dataDir <- normalizePath(file.path("..", "..", "data"))
forceUpdateAll <- FALSE

1 Objectives

The objectives of this notebook are to analyze the results from the first follow up round of the Rwanda long term soil health study.

2 Key Takeaways

Coming soon!

3 Data Prep

I’m going to load the baseline data from the baseline analysis. The report and data can be found here. I’ll load the new data directly from CommCare. The original baseline data object was d but I’m going to make it b. Each subsequent round will be r1, r2 and so on.

Overall I want to bring in 3 data sources:

  • Basline survey data and soil data
  • Round 1 survey and and soil data from 16B
  • Round 1 yield and soil data - these data come from paired climbing bean harvest measurements and soil samples from 16B
  • We can also look at maize paired yield and soil samples from 17A.

3.1 Baseline data

baselineDir <- normalizePath(file.path("..", "rw_baseline", "data"))
load(file=paste0(baselineDir, "/shs rw baseline full soil.Rdata")) # obj d
b <- baseVars

Context point: The baseline data has 2439 rows. This is 9 fewer rows than we expected in the baseline. This is because of some farmers not being surveyed as expected. See the baseline report for more details. Also, these baesline values have te

Alex Villec wrote a cleaning script to deal with the first round of Rwanda SHS follow up data and make key adjustments to the data. To utilize that do file here, I’m going to download the data from Commcare, save it, and have the dofile access that file to execute. However, the original file Alex was using had different variable names than the file pulled by the API. The options from here are to just go with the file from Alex or to align the variable names between his version and the CC version. It’s valuable to have the data directly from CC but it’ll involve more work up front

3.2 Round 1 data

source("../oaflib/commcareExport.R")
r <- getFormData("oafrwanda", "M&E", "16B Ubutaka (Soil)", forceUpdate = forceUpdateAll)
[1] "found fdd434a62c6512b320a4cb8c4fb872a"
write.csv(r, file="rawCcR1Data.csv", row.names = F)

3.3 Yield data

yp <- getFormData("oafrwanda", "M&E", "16B ALL Isarura (Harvest)", forceUpdate = forceUpdateAll)
[1] "found 736b25426bb4f9320a07d9c42b738ea"
write.csv(yp, file="rawCcYpData.csv", row.names=F)

The first round of data from CommCare has 2381 observations. This leaves XX number of farmers unsurveyed in the first survey round. See this cleaning file for more information on the farmers we did not find again in the first follow up.

Here I’m going to call the STATA cleaning file to make AV’s changes to the R1 follow up data. This requires that the data from CC have the same variable names as the STATA cleaning file. I’m going to try to execute that here:

stataDir <- normalizePath(file.path("..", "rw_round_1_check"))

Here I access the soil predictions from the OAF soil lab. Patrick Bell manages the lab and Mike Barber oversees the prediction scripts.

soilDir <- normalizePath(file.path("..", "..", "OAF Soil Lab Folder", "Projects", "rw_shs_second_round", "4_predicted", "other_summaries"))
soil <- read.csv(file=paste(soilDir, "combined-predictions-including-bad-ones.csv", sep = "/"))

3.4 Combine baseline and R1

Combine the available data by farmer and resolve merging issues. These data can be combined long as long as the variable names are consistent or wide. I’m going to combine the data long and use split type commands to aggregate the data more easily. Confirm the variable names are consistent. By advancing this code on 5/9/17, I’m for the time being ignoring the cleaning Alex did in his do file. I’ll need to go back and incorporate those changes.

TODO: see if the variables names in Alex’s raw data, shared by Nathaniel, match the data I’m downloading from commcare. If so, don’t use the var_names.xlsx sheet and instead use those variable names and Alex’s do file to preserve all of his changes.

Not many of the names are the same. I’ve downloaded the meta data from CommCare which I’ll use to simplify the cleaning of the round 1 data. I’m also going to reshape the baseline variable names to simplify the matching of baseline variables to round 1 variables.

datNames <- function(dat){
  varNames = names(dat)
  exVal = do.call(rbind, lapply(varNames, function(x){
    val = dat[1:3,x]
    return(val)
  }))
  
  out = cbind(varNames, exVal)
  return(out)
}
baseNames <- datNames(b)
write.csv(baseNames, file="baseline var names.csv", row.names = F)

Load Alex’s raw data and take the variable names from this. If I can align these variable names with the data from CC I can then execute Alex’s cleaning script on the CC data and proceed with combining the data

3.5 Stata .do file

rawDir <- normalizePath(file.path("Soil health study (year one)", "data"))
avRaw <- read.csv(paste(rawDir, "y1_shs_rwanda_28sep.csv", sep = "/"), stringsAsFactors = F)

It looks like the data from CommCare aligns with the raw data Alex worked with at info_formid which is the second index for avRaw and the 10th index for r. Let’s just try transferring them over and the work of updating the variable names through the CC codebook export may not be necessary!

varTest <- data.frame(fromcc = names(r)[10:409], fromav = names(avRaw)[2:401])
# head(varTest)
# tail(varTest)
#varTest[90:120,]

It seems to line up okay (with some adjustments)! To incorporate Alex’s cleaning code I have to export the data from R to a form Stata accept, run the code, and then load the data back in.

This function will remove all strange outputs from the data from CommCare so that the STATA code works

charClean <- function(df){
  
  df <- as.data.frame(lapply(df, function(x){
  x = gsub("'", '', x)
  x = gsub("^b", '', x)
  x = ifelse(grepl("map object", x)==T, NA, x)
  return(x)
  }))
return(df)
}
r <- charClean(r)
names(r)[10:409] <- names(avRaw)[2:401]
#export so stata can run - check for variable names longer than 32char
table(nchar(names(r)))

 2  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 32 33 34 36 37 38 39 40 41 42 43 44 45 46 47 48 49 51 52 
 1  4  3  1  1  2  6  1  1  2  3  5 17 11 16 12  5  8  1  7  1  3  9  9  3  7  2  3  1 28 16 47 32 11  7 27 18 21 31 10  7  4  3  1  1 
write.csv(r, file="toBeCleanedStata.csv", row.names = F)
stata("cleans_y1_shs_rwanda.do", stata.echo=F)

Now load the result of the Stata file

r <- read.csv("cleanedforR.csv", stringsAsFactors = F)
newName <- read_excel("var_names.xlsx", sheet=1)

qTypes <- c("Multiple Choice", "Phone Number or Numeric ID", "Checkbox", "Text", "Decimal", "Image Capture", "Barcode Scan", "GPS", "Integer", "Time", "Date")

newName <- newName %>% dplyr::select(1:6
) %>% dplyr::filter(newName$Type %in% qTypes) %>% as.data.frame()

#newName <- newName %>% filter(new.var.name!="general.comment")
metaVars <- names(r)[1:10]
newNameVars <- c(metaVars, newName$new.var.name)

length(newNameVars)==dim(r)[2]

write.csv(newNameVars, file="newVar check.csv")
write.csv(names(r), file="round1 Var check.csv")
names(r) <- newNameVars

# drop vars with drop
r <- r[,-which(grepl("drop.", names(r)))]
qNum <- c("Phone Number or Numeric ID", "Decimal", "Integer")
nums <- newName[newName$Type %in% qNum, "new.var.name"]
nums <- nums[-which(grepl("drop.", nums))]

toRemove <- c("phone", "oafid")
nums <- nums[!nums %in% toRemove]

# add in plot.size
#nums <- c(nums, "plot.size")

r[, nums] <- as.data.frame(lapply(r[,nums], function(x){
  as.numeric(as.character(x))
}))

4 Cleaning

The r dataframe has many more variables than the baseline survey. This was in part expected; we added questions to the first follow up round based on lessons from the baseline. It’s also due to how the survey was set up in CommCare. Before combining the baseline and the first follow up round I need to:

  • reshape the round 1 variables so that they appropriately match the baseline variables
  • Clean those variales or prepare them as need be for a
  • For variables with no match, clean
multiplot <- function(..., plotlist=NULL, file, cols=1, layout=NULL) {
  library(grid)
  # Make a list from the ... arguments and plotlist
  plots <- c(list(...), plotlist)
  numPlots = length(plots)
  # If layout is NULL, then use 'cols' to determine layout
  if (is.null(layout)) {
    # Make the panel
    # ncol: Number of columns of plots
    # nrow: Number of rows needed, calculated from # of cols
    layout <- matrix(seq(1, cols * ceiling(numPlots/cols)),
                    ncol = cols, nrow = ceiling(numPlots/cols))
  }
 if (numPlots==1) {
    print(plots[[1]])
  } else {
    # Set up the page
    grid.newpage()
    pushViewport(viewport(layout = grid.layout(nrow(layout), ncol(layout))))
    # Make each plot, in the correct location
    for (i in 1:numPlots) {
      # Get the i,j matrix positions of the regions that contain this subplot
      matchidx <- as.data.frame(which(layout == i, arr.ind = TRUE))
      print(plots[[i]], vp = viewport(layout.pos.row = matchidx$row,
                                      layout.pos.col = matchidx$col))
    }
  }
}

4.1 Drop variables

toDrop <- c("appformid", "id", "domain", "metadatadeviceid")
r <- r[,!names(r) %in% toDrop]
source("../oaflib/misc.R")
names(r) <- gsub("^y1_|intro_", "", names(r))
r[r=="."] <- NA
r <- divideGps(r, "gps_coord")

4.2 Categorical variables

The responses of the categorical variables should be regulated through CC, however, to check, make a table that shows the top ten responses in descending order and make a graph of response counts to know what to check. I’ll then capture any characters that should be numeric and convert them.

catVars <- names(r)[sapply(r, function(x){
  is.character(x)
})]
enumClean <- function(dat, x, toRemove){
  dat[,x] <- ifelse(dat[,x] %in% toRemove, NA, dat[,x])
  return(dat[,x])
}
strTable <- function(dat, x){
  varName = x
  tab = as.data.frame(table(dat[,x], useNA = 'ifany'))
  tab = tab[order(tab$Freq, decreasing = T),]
  end = ifelse(length(tab$Var1)<10, length(tab$Var1), 10)
  repOrder = paste(tab$Var1[1:end], collapse=", ")
  out = data.frame(variable = varName,
                   responses = repOrder)
  
  return(out)
}
# clean up known values
catEnumVals <- c("-99", "-88", "- 99", "-99.0", "88", "_88", "- 88", "0.88",
                 "--88", "__88", "-88.0", "99.0")
r[,catVars] <- sapply(catVars, function(y){
  r[,y] <- enumClean(r,y, catEnumVals)
})
responseTable <- do.call(rbind, lapply(catVars, function(x){
  strTable(r, x)
}))

4.2.1 Categorical response table

A simple table to preview the values in the data. The values are ranked by frequency.

kable(responseTable)
variable responses
metadatauserid c3e5e4d69726a6587d9d5739f3961b03, ab7675956342e27f3a134b45731ca6f9, a8f48eb2ccc435935cdefec31a49f512, 2da910f9aa814b352b62821db7ac30fc, 7e1b7bc7a7147b9f4ddfedab54e8e470, 43ab9369b7e43edaa7d9614594f4d1dd, 9938a37f596038d85181e4d38cff2433, bfb7f31368600aefe2c4386ad49c5126, 4a69416450e53b6e762ea707aaf80104, 089ae26df7d5ea3886dbbe3709c34013
metadatausername umushakashatsi, umushakashatsi3, umushakashatsi72, umushakashatsi42, umushakashatsi58, umushakashatsi14, umushakashatsi66, umushakashatsi7, umushakashatsi13, umushakashatsi73
metadatatimestart 2012-01-01T02:07:31.468000, 2012-01-01T21:53:26.687000, 2012-01-01T23:04:56.746000, 2012-01-06T20:14:52.707000, 2012-01-06T21:14:58.517000, 2012-01-07T01:08:44.167000, 2016-07-27T07:53:43.734000, 2016-07-27T08:39:53.902000, 2016-07-27T08:39:57.777000, 2016-07-27T08:41:57.353000
metadatatimeend 2012-01-06T20:52:59.887000, 2012-01-07T19:01:49.301000, 2012-01-07T19:04:31.323000, 2012-01-07T19:09:38.384000, 2016-07-27T09:41:47.415000, 2016-07-27T09:57:48.152000, 2016-07-27T10:43:47.085000, 2016-07-27T11:24:53.338000, 2016-07-27T11:25:03.144000, 2016-07-27T11:26:55.594000
start_time 09:00:00.000+02, 08:30:00.000+02, 09:40:00.000+02, 10:13:00.000+02, 10:36:00.000+02, 12:20:00.000+02, 09:14:00.000+02, 09:29:00.000+02, 10:14:00.000+02, 10:56:00.000+02
date 2016-08-10, 2016-08-11, 2016-08-08, 2016-08-17, 2016-08-03, 2016-08-18, 2016-08-22, 2016-08-19, 2016-08-04, 2016-08-12
enum_name Hagenimana bienvenue, MUCYOWIMIHIGO J MV, Nyandwi Anathalie, ZIMUKWIYE Dominique, Nyirangirimana jeanne, Torero pacifique, Utamuriza Jeanne, Niyidufasha nathanael, Rukundo japhet, NYIRAMPANO Bernadette
photo , 1325376816129.jpg, 1325447804135.jpg, 1325452024080.jpg, 1325873951716.jpg, 1325877535600.jpg, 1325891580194.jpg, 1469601919598.jpg, 1469601990645.jpg, 1469602247216.jpg
district Rutsiro, Karongi, Mugonero, Nyamasheke, Huye, Rwamagana, Gatsibo_NLWH, Gatsibo_LWH, Nyamagabe, Kayonza
cell_field Rubumba, Mubuga, Nyabicwamba, NYAGATARE, Mugera, MutongoCA, Bihumbe, Busetsa, Gihumuza, Kibyagira A
village Gasharu, Murambi, Rugarama, Kabeza, Karambo, Kigarama, Nyabugogo, Kabuga, Kivumu, Gasagara
farmer_list Havugimana celestin, Karekezi Celestin, Mukabinyange cecile, Mukafundi Marie, Musabyimana Jean, Ndananiwe Francois, Ndayambaje Emmanuel, Nsengiyumva Augustin, Nyirahabimana seraphine, Nyiraminani Constasie
farmer_respond NA, Akimana Jeannette, BIMENYANDE Djumapri, Habimana Emmanuel, Hagumagatsi Gaspard, Karekezi Celestin, Mukabinyange cecile, Mukangiriye Donatha, Mukankusi Beatrice, MUNYENSANGA Emmanuel
farmer_phonenumber NA, Ntayo, 0, ntayo, Nta telephone afite, Ntayo afite, 0.0, -, nta telephone afite, Ntayo bafite
d_phone NA, 0, Ntayo, ntayo, Ni wewabajijwe, -, Ntayo afite, O, Nta telephone afite, Ntayo bafite
neighbor_phonenumber NA, ntayo, 0, Ntayo, 0.0, -, 0789699430, 0785275883, 7.85275883E8, 0723071668
gender female, male
n_tubura_season not_a_client_3seasons, 16a 16b 17a, 16a 17a, 17a, 16a 16b, 16a, NA, 16b 17a, 16b, 16a not_a_client_3seasons
which_crop_16a_1 gor
which_maize_seed_16a_1 NA, gor_nsp, new_hybrid, OPV_saved, Hybride_saved, OPV_new
which_crop_16a_2 NA, yum, gor, big, insina, jum, soya, ray, shy, shaz
which_maize_seed_16a_2 NA, gor_nsp, Hybride, OPV_saved, OPV_new, Hybride_saved
fert_type1_16a None, DAP, NA, NPK-17, urea, NPK-22, npk2555
fert_type2_16a NA, urea, None, DAP, NPK-17, NPK-22, npk2555
quality_compost_16a Good, NA, Average, Bad
type_compost_16a cow, NA, goat, pig, other, plant, kitchen_waste, human, chicken
d_lime_16a no_lime, NA, lime_outside, lime_tubura, both_tubura_non_tubura
which_crop_16b_1 big, shy, saka, NA, jum, soya, gor, ray, nyo, yum
which_maize_seed_16b_1 NA, new_hybrid, gor_nsp, OPV_new, Hybride_saved, OPV_saved
which_crop_16b_2 NA, gor, yum, jum, insina, big, soya, saka, shy, ray
which_maize_seed_16b_2 NA, new_hybrid, OPV_new, gor_nsp, Hybride_saved, OPV_saved
fert_type1_16b None, NA, DAP, NPK-17, urea, NPK-22, npk2555
fert_type2_16b NA, None, urea, DAP, NPK-17
quality_compost_16b NA, Good, Average, Bad
type_compost_16b NA, cow, pig, goat, kitchen_waste, plant, human, other, chicken
d_lime_16b no_lime, NA, lime_outside, lime_tubura
how_use_residues feed_animals, mulching, leave_field, compost_use, burn_field, burn_discard, sell
field_texture clay_loam, loam, silty_clay_loam, sandy_clay_loam, sandy_loam, silty_loam, silty_clay, loamy_sand, sand, clay
field_erosion drainageditch, nothing, radicalterrace, gradualterrace
crop_direction not_applicable, NA, across_slope, down_slope
comments , Ntakibazo, ntakibazo, ntayo, Ntayo, Ntazo, ntazo, Ntakibazo., Ntacyahindutse, NA
sample_id 12, 137, 1503, 2044C, 2278, 2299, 2610, 2612, 2612C, 10
kg_yield_hwag_16b_1 NA
kg_seed_ananas_16b_2 NA
kg_seed_veg_16a_1 NA
kg_seed_16a_1 N, 1, 0, 2, -, 3, 4, 5, 6, 8
kg_seed_16a_2 , NA, 0.5, 1.0, 0.25, 2.0, 3.0, 1.5, 4.0, 5.0
kg_seed_16b_1 NA, , 3.0, 2.0, 1.0, 0.5, 1.5, 4.0, 5.0, 6.0
kg_seed_16b_2 , NA, 0.5, 1.0, 0.25, 2.0, 1.5, 3.0, 4.0, 5.0
kg_yield_16a_1 NA, 50.0, 20.0, 100.0, 30.0, 10.0, 40.0, 15.0, 200.0, 5.0
kg_yield_16a_2 , NA, 20.0, 10.0, 50.0, 30.0, 0.0, 15.0, 5.0, 100.0
kg_yield_16b_1 , NA, 20.0, 30.0, 10.0, 15.0, 5.0, 50.0, 40.0, 100.0
kg_yield_16b_2 , NA, 0.0, 10.0, 5.0, 20.0, 15.0, 3.0, 40.0, 50.0
gps_coord NA, , -1.5578864555610237 30.39436791689242 1525.93 15.0, -1.5631940702424174 30.227211802604916 1659.67 15.0, -1.5639320092237632 30.227385933820276 1434.79 10.0, -1.5667398240763533 30.273551799148027 979.26 10.0, -1.567033053159622 30.277914044142907 982.39 10.0, -1.5671285398447943 30.275353919885177 560.94 10.0, -1.5685424850437755 30.248542080122405 1468.14 20.0, -1.5688621725334673 30.24841864727349 851.74 10.0
unique_location Gatsibo_NLWH2610, Gatsibo_NLWH2612, Gatsibo_NLWH2612C, Huye137, Karongi1503, Rutsiro2044C, Rutsiro2278, Rutsiro2299, Gatsibo_LWH2476, Gatsibo_LWH2476C

4.2.2 Categorical response graphs

repGraphs <- function(dat, x){
  tab = as.data.frame(table(dat[,x], useNA = 'ifany'))
  tab = tab[order(tab$Freq, decreasing = T),]
  print(
    ggplot(data=tab, aes(x=Var1, y=Freq)) + geom_bar(stat="identity") +
      theme(legend.position = "bottom", axis.text.x = element_text(angle = 45, hjust = 1)) +
      labs(title =paste0("Composition of variable: ", x))
  )
}
adminVars <- c(names(r)[grep("meta", names(r))], "start_time", "enum_name", "photo", "cell_field", "village", "farmer_respond", "farmer_phonenumber", "d_phone", "neighbor_phonenumber", "farmer_list", "unique_location", "comments", "gps_coord", "sample_id")
nonAdminVars <- catVars[!catVars %in% adminVars]
for(i in 1:length(nonAdminVars)){
  repGraphs(r, nonAdminVars[i])
}

4.2.3 Manual character cleaning

r$female <- ifelse(r$gender=="female", 1, 0)
r$district <- ifelse(grepl("nyanza", r$district)==T, "Nyanza", r$district)
table(r$kg_seed_16b_1)

        0.0 0.125  0.25   0.3   0.4   0.5  0.75   1.0  1.25   1.5  10.0  10.5 100.0  12.0  12.5 120.0  13.0  14.0  15.0 150.0  16.0  18.0  19.0  19.5 
  354     1     3    18     1     1   148     3   245     2   140    41     5     3    15     1     2     1     4    21     1     1     2     1     1 
  2.0   2.5  20.0 200.0  21.0  22.0  22.5  24.0  25.0   3.0   3.5  3.75  30.0 300.0  33.0  35.0  37.5  39.0   4.0   4.5  40.0  45.0   5.0   5.5  50.0 
  248    24    12     1     2     1     1     1     8   261     4     1    12     1     1     1     1     1    89    54     5     1    70     1     5 
500.0   6.0   6.5  60.0   7.0   7.5  70.0   8.0  80.0   9.0  9.75 
    1    68     3     1    26    23     3    30     1    16     1 
table(r$kg_yield_16a_2)

          0.0   0.25    0.5    1.0    1.5   10.0  100.0 1000.0  105.0   11.0   12.0  120.0 1250.0   13.0   15.0   15.5  150.0   16.0   16.5  165.0   17.0 
  1352     47      1      4     11      3     63     31      1      2      2      9      5      1      2     45      1      7      2      1      1      1 
   2.0    2.5   20.0  200.0   22.0   22.5   23.0   24.0  240.0   25.0   26.0   28.0    3.0   30.0  300.0   32.0   35.0  350.0   36.0   37.5  375.0   39.0 
    20      2     79      9      1      7      1      1      1     11      1      1     29     53      4      2      1      1      1      1      1      1 
   4.0    4.5   40.0  400.0   45.0   48.0    5.0   50.0  500.0   53.0   55.0    6.0    6.5   60.0  600.0  630.0    7.0    7.5   70.0   75.0    8.0   80.0 
    16     12     24      4      3      2     43     63      1      1      1     12      1     18      1      1      7     11      7      1     11     11 
  86.0    9.0   90.0 
     1      5      4 
strtoNum <- c("kg_seed_16b_1", "kg_yield_16a_1", "kg_yield_16b_1", "kg_yield_16b_2")
r[,strtoNum] <- sapply(r[,strtoNum], function(x){as.numeric(x)})

Notes on the categorical variables:

  • We don’t have many actual responses on seed type despite all farmers telling us about a crop they are growing. Why? Check that there wasn’t a mislabeling of variables.
  • Check the ‘which_maize_seed’ variables to make certain they’re flexible to the type of crop selected in the previous question.
  • Confirm that blank is NA not 0.

4.3 Numeric variables

numVars <- names(r)[sapply(r, function(x){
  is.numeric(x)
})]

Basic cleaning of known issues like enumerator codes for DK, NWR, etc.

enumVals <- c(-88,-85, -99)
r[,numVars] <- sapply(numVars, function(y){
  r[,y] <- enumClean(r,y, enumVals)
})

4.3.1 Numeric outlier table

iqr.check <- function(dat, x) { 
  q1 = summary(dat[,x])[[2]]
  q3 = summary(dat[,x])[[5]] 
  iqr = q3-q1
  mark  = ifelse(dat[,x] < (q1 - (1.5*iqr)) | dat[,x] > (q3 + (1.5*iqr)), 1,0)
  tab = rbind(
    summary(dat[,x]),
    summary(dat[mark==0, x])
  )
  return(tab)
}
# remove admin vars
numAdminVars <- c(numVars[1:3])
numVarsNotAdmin <- numVars[!numVars %in% numAdminVars]
iqrTab <- do.call(plyr::rbind.fill, lapply(numVarsNotAdmin, function(y){
  #print(y)
  res = iqr.check(r, y)
  #print(dim(res))
  out = data.frame(var=rbind(y, paste(y, ".iqr", sep="")), res)
  return(out)
}))
iqrTab[,2:8] <- sapply(iqrTab[,2:8], function(x){round(x,1)})

The outlier table summarizes the numeric variables with and without IQR outliers to show how the data changes based on this filter.

knitr::kable(iqrTab, row.names = F, digits = 0, format = 'html')
var Min. X1st.Qu. Median Mean X3rd.Qu. Max. NA.s
d_client_16b 0 0 0 0 1 1 NA
d_client_16b.iqr 0 0 0 0 1 1 NA
d_client_17a 0 0 0 0 1 1 NA
d_client_17a.iqr 0 0 0 0 1 1 NA
age 16 35 45 47 57 90 NA
age.iqr 16 35 45 47 57 90 NA
n_household 0 4 5 5 7 39 NA
n_household.iqr 0 4 5 5 7 11 NA
n_cows 0 0 1 1 1 15 NA
n_cows.iqr 0 0 1 1 1 2 NA
n_goats 0 0 0 1 2 18 NA
n_goats.iqr 0 0 0 1 2 5 NA
n_chickens 0 0 0 1 1 40 NA
n_chickens.iqr 0 0 0 0 0 2 NA
n_pigs 0 0 0 0 1 11 NA
n_pigs.iqr 0 0 0 0 1 2 NA
n_sheep 0 0 0 0 0 35 NA
n_sheep.iqr 0 0 0 0 0 0 NA
field_length 0 13 20 26 32 214 NA
field_length.iqr 0 13 20 23 30 60 NA
field_width 0 12 20 24 31 160 NA
field_width.iqr 0 12 20 22 30 59 NA
n_spots 3 3 3 4 5 5 NA
n_spots.iqr 3 3 3 4 5 5 NA
fert_kg1_16a 0 1 2 4 5 80 1408
fert_kg1_16a.iqr 0 1 2 3 4 11 1408
fert_kg2_16a 0 0 0 2 2 200 1198
fert_kg2_16a.iqr 0 0 0 1 2 5 1198
d_compost_16a 0 1 1 1 1 1 271
d_compost_16a.iqr 1 1 1 1 1 1 271
kg_compost_16a 0 100 200 268 300 20000 613
kg_compost_16a.iqr 0 100 191 205 300 600 613
kg_lime_16a 0 15 40 66 100 500 2345
kg_lime_16a.iqr 0 10 25 52 100 150 2345
fert_kg1_16b 0 1 2 4 4 100 1964
fert_kg1_16b.iqr 0 1 2 2 3 8 1964
fert_kg2_16b 0 0 0 0 0 88 1656
fert_kg2_16b.iqr 0 0 0 0 0 0 1656
d_compost_16b 0 0 1 0 1 1 529
d_compost_16b.iqr 0 0 1 0 1 1 529
kg_compost_16b 0 100 160 238 300 10000 1411
kg_compost_16b.iqr 0 100 150 193 250 600 1411
kg_lime_16b 1 10 25 59 50 650 2353
kg_lime_16b.iqr 1 10 25 32 50 100 2353
field_slope -5 3 6 9 14 60 NA
field_slope.iqr -5 3 6 9 14 30 NA
field_n_crops 0 1 1 2 2 30 343
field_n_crops.iqr 0 1 1 1 2 3 343
kg_seed_16b_1 0 1 2 5 4 500 754
kg_seed_16b_1.iqr 0 1 2 3 4 10 754
kg_yield_16a_1 0 15 34 73 80 6000 1570
kg_yield_16a_1.iqr 0 12 30 41 50 170 1570
kg_yield_16b_1 0 8 20 53 50 6000 600
kg_yield_16b_1.iqr 0 8 20 28 40 112 600
kg_yield_16b_2 0 3 10 25 25 600 1954
kg_yield_16b_2.iqr 0 3 8 13 20 55 1954
yield_compare_16a_1 1 1 1 2 3 3 1506
yield_compare_16a_1.iqr 1 1 1 2 3 3 1506
yield_compare_16a_2 1 1 2 2 2 3 1355
yield_compare_16a_2.iqr 1 1 2 2 2 3 1355
yield_compare_16b_1 1 1 1 2 2 3 358
yield_compare_16b_1.iqr 1 1 1 2 2 3 358
yield_compare_16b_2 1 1 1 2 2 3 1734
yield_compare_16b_2.iqr 1 1 1 2 2 3 1734
lat -3 -2 -2 -2 -2 -2 497
lat.iqr -3 -2 -2 -2 -2 -2 497
lon 29 29 30 30 30 31 497
lon.iqr 29 29 30 30 30 31 497
alt -108 1513 1673 1668 1887 2668 497
alt.iqr 957 1541 1680 1728 1887 2430 497
precision 5 10 15 19 15 4181 497
precision.iqr 5 10 15 13 15 20 497
female 0 0 1 1 1 1 NA
female.iqr 0 0 1 1 1 1 NA

4.3.2 Outlier Graphs

# http://rforpublichealth.blogspot.com/2014/02/ggplot2-cheatsheet-for-visualizing.html
for(i in 1:length(numVarsNotAdmin)){
    base <- ggplot(r, aes(x=r[,numVarsNotAdmin[i]])) + labs(x = numVarsNotAdmin[i])
    temp1 <- base + geom_density()
    temp2 <- base + geom_histogram()
    #temp2 <- boxplot(r[,numVars[i]],main=paste0("Variable: ", numVars[i]))
    multiplot(temp1, temp2, cols = 2)
}

4.4 Check for unique ids

I’m seeing that there are duplicated farmers in the data when I’m trying to reshape the r data from wide to long. Let’s check them out here and see if we can figure out which observation is right.

  • Check Alex’s do file to see if there’s mention of these farmers. [No mention]
  • Check the baseline values as these should line up.
length(r$sample_id)==length(unique(r$sample_id))
[1] FALSE
dups <- r$sample_id[duplicated(r$sample_id)]
dupIndex <- which(duplicated(r$sample_id))
#dupDat <- r[r$sample_id %in% dups,]
head(r[r$sample_id==dups[1],])
head(r[r$sample_id==dups[2],])

Let’s solve the unique id issue by looking at identifying information in the baseline data

roundId <- r %>%
  dplyr::select(district, cell_field, village, sample_id, farmer_list) %>%
  filter(r$sample_id %in% dups)
#d
load("rawBaselineWithIdentifers.Rdata")
baseId <- d %>% 
  dplyr::select(district, selected_cell, umudugudu,  sample_id, farmer_name ) %>%
  filter(d$sample_id %in% dups)
#baseId
roundId

4.4.1 Correct duplicates

Correct the duplicates I can and drop the others for now. Flag the duplicated ones and save them to share with Nathaniel.

TODO(mattlowes) - share any remaining duplicates with Nathaniel and see if he has a solution. Also see if he can understand why this might have happened and if they should actually have a different sample id.

r <- r %>% mutate(
    dup = ifelse(
      sample_id == "12" & cell_field == "MUNANIRA" |
      sample_id == "137" & village == "Rusuma" |
      sample_id == "1503" & farmer_list=="NAKAGIZE Val\\xc3\\xa9rie" |
      #sample_id == "2044C" &  # same!
      sample_id == "2278" & cell_field=="Nkira A" | # check this as maybe this was the only thing wrong?
      #sample_id == "2299" & # same!
      sample_id == "2610" & village=="agakiri" #|  #agakiri is close to gakiri in spelling. Is this just a typo?
      #sample_id == "2612" &  # same names!
      #sample_id == "2612C" # same names!
      , 1, 0)
) %>% filter(
  dup!=1
) %>% dplyr::select(-dup) 
# run this code again from above to get updated duplicates list
#length(r$sample_id)==length(unique(r$sample_id))
dups <- r$sample_id[duplicated(r$sample_id)]
dupIndex <- which(duplicated(r$sample_id))
# for the time being drop the observations that are duplicates
r <- r[!r$sample_id %in% dups,]

4.5 Reshape variables

This should include the baseline variables as well.

Let’s first check with the baseline data to see what variables we made there so I can make the same ones from the round 1 data. There are some variables that are baseline variables only like variables asking about historical practices. There are then other variables that will vary by season. These are the variables that we ultimately want in to shape in a long dataset by season to analyze changes overtime in practices and soil management. I think this will result in a dataset that has one row per farmer per season. Some variables may not fit nicely into this but we can deal with those. For variables that aren’t changing over time they’ll show as not important in our model. They’re important for matching farmers.

There are a lot of variables to try to line up. Some already have the same name but how to best combine the ones that have different variable names? I’m going to write a function that takes a variable name from b and a variable name from r that should go together, updates the r variable name and uses that info to rbind the data into a long dataset.

# names(b)
# names(r)
# check the names that already match
baselineFound <- names(b)[names(b) %in% names(r)] # not many variable names are aligned

Update variable names so that any variable with 16a or 16b has a the a or b season designation at the end it so I can replicate the gather() and spread() options for reorganizing the data by season and by plot. This means that the variable names will retain their designation of first or second application and be distinguishable.

TODO(mattlowes) - rename the variables according to that convention to reshape the r data. Keep the baseline data in mind as we’ll want to do the same thing with the baseline data to make them match.

r <- r %>% rename(
  which_crop_1_16a = which_crop_16a_1,
  which_maize_seed_1_16a = which_maize_seed_16a_1,
  which_crop_2_16a = which_crop_16a_2,
  which_maize_seed_2_16a = which_maize_seed_16a_2,
  kg_seed_veg_1_16a = kg_seed_veg_16a_1,
  kg_seed_1_16a = kg_seed_16a_1,
  kg_seed_2_16a = kg_seed_16a_2,
  kg_yield_1_16a = kg_yield_16a_1,
  kg_yield_2_16a = kg_yield_16a_2,
  yield_compare_1_16a = yield_compare_16a_1,
  yield_compare_2_16a = yield_compare_16a_2,
  
  which_crop_1_16b = which_crop_16b_1,
  which_maize_seed_1_16b = which_maize_seed_16b_1,
  which_crop_2_16b = which_crop_16b_2,
  which_maize_seed_2_16b = which_maize_seed_16b_2,
  #kg_seed_veg_1_16a = kg_seed_veg_16a_1,
  kg_seed_1_16b = kg_seed_16b_1,
  kg_seed_2_16b = kg_seed_16b_2,
  kg_yield_1_16b = kg_yield_16b_1,
  kg_yield_2_16b = kg_yield_16b_2,
  yield_compare_1_16b = yield_compare_16b_1,
  yield_compare_2_16b = yield_compare_16b_2
)
aSeason <- names(r)[grep("16a", names(r))]
bSeason <- names(r)[grep("16b", names(r))]
seasonalVars <- rbind(aSeason, bSeason, "sample_id")
farmerVars <- names(r)[!names(r) %in% seasonalVars]
# example data
# df <- data.frame(
#   id = 1:10,
#   time = as.Date('2009-01-01') + 0:9,
#   Q3.2.1. = rnorm(10, 0, 1),
#   Q3.2.2. = rnorm(10, 0, 1),
#   Q3.2.3. = rnorm(10, 0, 1),
#   Q3.3.1. = rnorm(10, 0, 1),
#   Q3.3.2. = rnorm(10, 0, 1),
#   Q3.3.3. = rnorm(10, 0, 1)
# )
# 
# df %>%
#   gather(key, value, -id, -time) %>%
#   extract(key, c("question", "loop_number"), "(Q.\\..)\\.(.)") %>%
#   spread(question, value)
source("../oaflib/misc.R")
# aDat <- r[,names(r) %in% aSeason] # works for this too!
# aDat <- aDat[,grep("16a_1", names(aDat))] # works for this
aDat <- r[,names(r) %in% seasonalVars] # works for this!
#http://stackoverflow.com/questions/25925556/gather-multiple-sets-of-columns
seasonalDat <- aDat %>%
  gather(key, value, -sample_id) %>%
  tidyr::extract(key, c("variable", "season"), "(^.*\\_1.)(.)") %>%
  mutate(season = paste0("16", season)) %>% 
  spread(variable, value)
names(seasonalDat) <- gsub("_16", "", names(seasonalDat))

TODO(mattlowes) - confirm that the tidyr process worked as I expected as there are numerous missing values. These seem to appear where the variable only had one version of the variable, _16, rather than a _16a and a _16b. Check out how this is handling variables with _17 instead of _16.

4.6 Merge seasonal and demographic data

rs <- left_join(seasonalDat, r[,!names(r) %in% seasonalVars], by="sample_id")
Error: 'sample_id' column not found in lhs, cannot join

4.7 Create new variables

4.7.1 Field variables

rs$dim <- rs$field_length * rs$field_width
inputVars <- names(rs)[grep("fert_|quality_compost|type_compost|which_crop|which_maize", names(rs))]
rs[,inputVars] <- sapply(rs[, inputVars], tolower)
# input quanitites
rs$fert_kg_urea1 <- ifelse(rs$fert_type1=="urea", rs$fert_kg1, NA)
rs$fert_kg_urea2 <- ifelse(rs$fert_type2=="urea", rs$fert_kg2, NA)
rs$fert_total_urea <- apply(rs[, grep("(urea.)", names(rs))], 1, function(x){
  sum(as.numeric(x), na.rm=T)})
rs$fert_kg_dap1 <- ifelse(rs$fert_type1=="dap", rs$fert_kg1, NA)
rs$fert_kg_dap2 <- ifelse(rs$fert_type2=="dap", rs$fert_kg2, NA)
rs$fert_total_dap <- apply(rs[, grep("(dap.)", names(rs))], 1, function(x){
  sum(as.numeric(x), na.rm=T)})
rs$fert_kg_17npk1 <- ifelse(rs$fert_type1=="npk-17", rs$fert_kg1, NA)
rs$fert_kg_17npk2 <- ifelse(rs$fert_type2=="npk-17", rs$fert_kg2, NA)
rs$fert_total_npk17 <- apply(rs[, grep("(17npk.)", names(rs))], 1, function(x){
  sum(as.numeric(x), na.rm=T)})
rs$fert_kg_22npk1 <- ifelse(rs$fert_type1=="npk-22", rs$fert_kg1, NA)
rs$fert_kg_22npk2 <- ifelse(rs$fert_type2=="npk-22", rs$fert_kg2, NA)
rs$fert_total_npk22 <- apply(rs[, grep("(22npk.)", names(rs))], 1, function(x){
  sum(as.numeric(x), na.rm=T)})
rs$fert_kg_2555npk1 <- ifelse(rs$fert_type1=="npk2555", rs$fert_kg1, NA)
rs$fert_kg_2555npk2 <- ifelse(rs$fert_type2=="npk2555", rs$fert_kg2, NA)
rs$fert_total_npk2555 <- apply(rs[, grep("(2555npk.)", names(rs))], 1, function(x){
  sum(as.numeric(x), na.rm=T)})

4.7.2 Demographic variables

rs$season_16a <- ifelse(grep("16a", rs$n_tubura_season), 1, 0)
Error in `$<-.data.frame`(`*tmp*`, "season_16a", value = c(1, 1, 1, 1,  : 
  replacement has 2204 rows, data has 4762

4.8 Combine long with baseline

The matchRounds function updates variable names across rounds and reports the index and new name of the variables. I can then take the first part of the list for dat1 and the second part for dat2.

matchRounds <- function(dat1, dat2, var1, var2, new=NULL, choice="first"){
  
  
  
  if (choice=="first"){
    var2new  = var1
    #names(dat2)[names(dat2)==var2] <- var2new
    return(list(
      list(var1, grep(var1, names(dat1))),
      list(var2new, grep(var2, names(dat2)))
                ))
    
  } else if (choice=="second") {
    var1new = var2
    #names(dat1)[names(dat1)==var1] <- var1new
    return(list(
      list(var1new, grep(var1, names(dat1))),
      list(var2, grep(var2, names(dat2)))
                ))
    
  } else{
    var1new = var2new = new
    #names(dat2)[names(dat2)==var2] <- var2new 
    #names(dat1)[names(dat1)==var1] <- var1new
    return(list(
      list(var1new, grep(var1, names(dat1))),
      list(var2new, grep(var2, names(dat2)))
                ))
  }
} 


namesToUpdate <- rbind(
 c("demographicdate", "date", "first"),
  c("sample", "d_sample", "second")
)


# example
dat1=b
dat2=r
var1 = "field_dim1"
var2 = "field_length"
choice="first"

test <- matchRounds(b, r, "field_dim1", "field_length", choice="first")
test2 <- matchRounds(b, r, "field_dim2", "field_width", choice="first")


test <- lapply(namesToUpdate, function(x,y,z){
  val = matchRounds(b, r, x, y, choice=z)
  return(val)
})

5 Analysis

5.1 Demographic summary

5.2 Soil summary

5.3 Longitudinal soil summary

5.4 Matching

5.5 Regressions

6 Summary

6.1 Changes to the survey

7 Appendix

LS0tCnRpdGxlOiAiUndhbmRhIFNvaWwgSGVhbHRoIFN0dWR5IC0gUm91bmQgMSIKYXV0aG9yOiAnW01hdHQgTG93ZXNdKG1haWx0bzptYXR0Lmxvd2VzQG9uZWFjcmVmdW5kLm9yZyknCmRhdGU6ICdgciBmb3JtYXQoU3lzLnRpbWUoKSwgIiVCICVkLCAlWSIpYCcKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6CiAgICBudW1iZXJfc2VjdGlvbnM6IHllcwogICAgY29kZV9mb2xkaW5nOiBzaG93CiAgICB0aGVtZTogZmxhdGx5CiAgICB0b2M6IHllcwogICAgdG9jX2RlcHRoOiA2CiAgICB0b2NfZmxvYXQ6IHllcwotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQojIyMjIHNldCB1cAojIyBjbGVhciBlbnZpcm9ubWVudCBhbmQgY29uc29sZQpybShsaXN0ID0gbHMoKSkKY2F0KCJcMDE0IikKCiMjIHNldCB1cCBzb21lIGdsb2JhbCBvcHRpb25zCiMgYWx3YXlzIHNldCBzdHJpbmdzQXNGYWN0b3JzID0gRiB3aGVuIGxvYWRpbmcgZGF0YQpvcHRpb25zKHN0cmluZ3NBc0ZhY3RvcnM9RkFMU0UpCgojIHNob3cgdGhlIGNvZGUKa25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFKQoKIyBkZWZpbmUgYWxsIGtuaXRyIHRhYmxlcyB0byBiZSBodG1sIGZvcm1hdApvcHRpb25zKGtuaXRyLnRhYmxlLmZvcm1hdCA9ICdodG1sJykKCiMgY2hhbmdlIGNvZGUgY2h1bmsgZGVmYXVsdCB0byBub3Qgc2hvdyB3YXJuaW5ncyBvciBtZXNzYWdlcwprbml0cjo6b3B0c19jaHVuayRzZXQod2FybmluZyA9IEZBTFNFLCBtZXNzYWdlID0gRkFMU0UpCgojIyBsb2FkIGxpYnJhcmllcwojIGRwbHlyIGFuZCB0aWJibGUgYXJlIGZvciB3b3JraW5nIHdpdGggdGFibGVzCiMgcmVzaGFwZSBpcyBmb3IgZWFzeSB0YWJsZSB0cmFuc2Zvcm1hdGlvbgojIGtuaXRyIGlzIHRvIG1ha2UgcHJldHR5IHRhYmxlcyBhdCB0aGUgZW5kCiMgZ2dwbG90MiBpcyBmb3IgbWFraW5nIGdyYXBocwojIHJlYWR4bCBpcyBmb3IgcmVhZGluZyBpbiBFeGNlbCBmaWxlcwojIE1BU1MgaXMgZm9yIHJ1bm5pbmcgYm94Y294IHRlc3RzCiMgZ3JpZEV4dHJhIGlzIGZvciBhcnJhbmdpbmcgcGxvdHMKIyBjb3dwbG90IGlzIGZvciBhZGRpbmcgc3VidGl0bGVzIHRvIHBsb3RzCiMgcm9idXN0YmFzZSBpcyB0byBydW4gcm9idXN0IHJlZ3Jlc3Npb25zIHRvIGNvbXBlbnNhdGUgZm9yIG91dGxpZXJzCiMgY2FyIGlzIGZvciBwZXJmb3JtaW5nIGxvZ2l0IHRyYW5zZm9ybWF0aW9ucwpsaWJzIDwtIGMoImRwbHlyIiwgInJlc2hhcGUyIiwgImtuaXRyIiwgImdncGxvdDIiLCAidGliYmxlIiwgInJlYWR4bCIsIAogICAgIk1BU1MiLCAiZ3JpZEV4dHJhIiwgImNvd3Bsb3QiLCAicm9idXN0YmFzZSIsICJjYXIiLCAiUlN0YXRhIiwgImZvcmVpZ24iLAogICAgInRpZHlyIikKbGFwcGx5KGxpYnMsIHJlcXVpcmUsIGNoYXJhY3Rlci5vbmx5ID0gVCwgcXVpZXRseSA9IFQsIHdhcm4uY29uZmxpY3RzID0gRikKCiMjIyMgZGVmaW5lIGhlbHBmdWwgZnVuY3Rpb25zCiMgZGVmaW5lIGZ1bmN0aW9uIHRvIGFkanVzdCB0YWJsZSB3aWR0aHMKaHRtbF90YWJsZV93aWR0aCA8LSBmdW5jdGlvbihrYWJsZV9vdXRwdXQsIHdpZHRoKSB7CiAgd2lkdGhfaHRtbCA8LSBwYXN0ZTAocGFzdGUwKCc8Y29sIHdpZHRoPSInLCB3aWR0aCwgJyI+JyksIGNvbGxhcHNlID0gIlxuIikKICBzdWIoIjx0YWJsZT4iLCBwYXN0ZTAoIjx0YWJsZT5cbiIsIHdpZHRoX2h0bWwpLCBrYWJsZV9vdXRwdXQpCn0KCm9wdGlvbnMoIlJTdGF0YS5TdGF0YVZlcnNpb24iID0gMTIpCm9wdGlvbnMoIlJTdGF0YS5TdGF0YVBhdGgiID0gIi9BcHBsaWNhdGlvbnMvU3RhdGEvU3RhdGFTRS5hcHAvQ29udGVudHMvTWFjT1Mvc3RhdGEtc2UiKQpgYGAKCmBgYHtyfQpkYXRhRGlyIDwtIG5vcm1hbGl6ZVBhdGgoZmlsZS5wYXRoKCIuLiIsICIuLiIsICJkYXRhIikpCmZvcmNlVXBkYXRlQWxsIDwtIEZBTFNFCmBgYAoKIyBPYmplY3RpdmVzCgpUaGUgb2JqZWN0aXZlcyBvZiB0aGlzIG5vdGVib29rIGFyZSB0byBhbmFseXplIHRoZSByZXN1bHRzIGZyb20gdGhlIGZpcnN0IGZvbGxvdyB1cCByb3VuZCBvZiB0aGUgUndhbmRhIGxvbmcgdGVybSBzb2lsIGhlYWx0aCBzdHVkeS4KCiMgS2V5IFRha2Vhd2F5cwoKPiBDb21pbmcgc29vbiEKCiMgRGF0YSBQcmVwCgpJJ20gZ29pbmcgdG8gbG9hZCB0aGUgYmFzZWxpbmUgZGF0YSBmcm9tIHRoZSBiYXNlbGluZSBhbmFseXNpcy4gVGhlIHJlcG9ydCBhbmQgZGF0YSBjYW4gYmUgZm91bmQgaGVyZS4gSSdsbCBsb2FkIHRoZSBuZXcgZGF0YSBkaXJlY3RseSBmcm9tIENvbW1DYXJlLiBUaGUgb3JpZ2luYWwgYmFzZWxpbmUgZGF0YSBvYmplY3Qgd2FzIGBkYCBidXQgSSdtIGdvaW5nIHRvIG1ha2UgaXQgYGJgLiBFYWNoIHN1YnNlcXVlbnQgcm91bmQgd2lsbCBiZSBgcjFgLCBgcjJgIGFuZCBzbyBvbi4KCk92ZXJhbGwgSSB3YW50IHRvIGJyaW5nIGluIDMgZGF0YSBzb3VyY2VzOgoKKiBCYXNsaW5lIHN1cnZleSBkYXRhIGFuZCBzb2lsIGRhdGEKKiBSb3VuZCAxIHN1cnZleSBhbmQgYW5kIHNvaWwgZGF0YSBmcm9tIDE2QgoqIFJvdW5kIDEgeWllbGQgYW5kIHNvaWwgZGF0YSAtIHRoZXNlIGRhdGEgY29tZSBmcm9tIHBhaXJlZCBjbGltYmluZyBiZWFuIGhhcnZlc3QgbWVhc3VyZW1lbnRzIGFuZCBzb2lsIHNhbXBsZXMgZnJvbSAxNkIKKiBXZSBjYW4gYWxzbyBsb29rIGF0IG1haXplIHBhaXJlZCB5aWVsZCBhbmQgc29pbCBzYW1wbGVzIGZyb20gMTdBLgoKIyMgQmFzZWxpbmUgZGF0YQoKYGBge3J9CmJhc2VsaW5lRGlyIDwtIG5vcm1hbGl6ZVBhdGgoZmlsZS5wYXRoKCIuLiIsICJyd19iYXNlbGluZSIsICJkYXRhIikpCgpsb2FkKGZpbGU9cGFzdGUwKGJhc2VsaW5lRGlyLCAiL3NocyBydyBiYXNlbGluZSBmdWxsIHNvaWwuUmRhdGEiKSkgIyBvYmogZApiIDwtIGJhc2VWYXJzCmBgYAoKKipDb250ZXh0IHBvaW50Kio6IFRoZSBiYXNlbGluZSBkYXRhIGhhcyBgciBkaW0oYilbMV1gIHJvd3MuIFRoaXMgaXMgYHIgMjQ0OC1kaW0oYilbMV1gIGZld2VyIHJvd3MgdGhhbiB3ZSBleHBlY3RlZCBpbiB0aGUgYmFzZWxpbmUuIFRoaXMgaXMgYmVjYXVzZSBvZiBzb21lIGZhcm1lcnMgbm90IGJlaW5nIHN1cnZleWVkIGFzIGV4cGVjdGVkLiBTZWUgdGhlIGJhc2VsaW5lIHJlcG9ydCBmb3IgbW9yZSBkZXRhaWxzLiBBbHNvLCB0aGVzZSBiYWVzbGluZSB2YWx1ZXMgaGF2ZSB0ZQoKW0FsZXggVmlsbGVjXShtYXRpbHRvOmFsZXgudmlsbGVjQG9uZWFjcmVmdW5kLm9yZykgd3JvdGUgYSBjbGVhbmluZyBzY3JpcHQgdG8gZGVhbCB3aXRoIHRoZSBmaXJzdCByb3VuZCBvZiBSd2FuZGEgU0hTIGZvbGxvdyB1cCBkYXRhIGFuZCBtYWtlIGtleSBhZGp1c3RtZW50cyB0byB0aGUgZGF0YS4gVG8gdXRpbGl6ZSB0aGF0IGRvIGZpbGUgaGVyZSwgSSdtIGdvaW5nIHRvIGRvd25sb2FkIHRoZSBkYXRhIGZyb20gQ29tbWNhcmUsIHNhdmUgaXQsIGFuZCBoYXZlIHRoZSBkb2ZpbGUgYWNjZXNzIHRoYXQgZmlsZSB0byBleGVjdXRlLiBIb3dldmVyLCB0aGUgb3JpZ2luYWwgZmlsZSBBbGV4IHdhcyB1c2luZyBoYWQgZGlmZmVyZW50IHZhcmlhYmxlIG5hbWVzIHRoYW4gdGhlIGZpbGUgcHVsbGVkIGJ5IHRoZSBBUEkuIFRoZSBvcHRpb25zIGZyb20gaGVyZSBhcmUgdG8ganVzdCBnbyB3aXRoIHRoZSBmaWxlIGZyb20gQWxleCBvciB0byBhbGlnbiB0aGUgdmFyaWFibGUgbmFtZXMgYmV0d2VlbiBoaXMgdmVyc2lvbiBhbmQgdGhlIENDIHZlcnNpb24uIEl0J3MgdmFsdWFibGUgdG8gaGF2ZSB0aGUgZGF0YSBkaXJlY3RseSBmcm9tIENDIGJ1dCBpdCdsbCBpbnZvbHZlIG1vcmUgd29yayB1cCBmcm9udAoKIyMgUm91bmQgMSBkYXRhCgpgYGB7cn0Kc291cmNlKCIuLi9vYWZsaWIvY29tbWNhcmVFeHBvcnQuUiIpCnIgPC0gZ2V0Rm9ybURhdGEoIm9hZnJ3YW5kYSIsICJNJkUiLCAiMTZCIFVidXRha2EgKFNvaWwpIiwgZm9yY2VVcGRhdGUgPSBmb3JjZVVwZGF0ZUFsbCkKd3JpdGUuY3N2KHIsIGZpbGU9InJhd0NjUjFEYXRhLmNzdiIsIHJvdy5uYW1lcyA9IEYpCmBgYAoKIyMgWWllbGQgZGF0YQoKYGBge3J9CnlwIDwtIGdldEZvcm1EYXRhKCJvYWZyd2FuZGEiLCAiTSZFIiwgIjE2QiBBTEwgSXNhcnVyYSAoSGFydmVzdCkiLCBmb3JjZVVwZGF0ZSA9IGZvcmNlVXBkYXRlQWxsKQp3cml0ZS5jc3YoeXAsIGZpbGU9InJhd0NjWXBEYXRhLmNzdiIsIHJvdy5uYW1lcz1GKQpgYGAKClRoZSBmaXJzdCByb3VuZCBvZiBkYXRhIGZyb20gQ29tbUNhcmUgaGFzIGByIGRpbShyKVsxXWAgb2JzZXJ2YXRpb25zLiBUaGlzIGxlYXZlcyBYWCBudW1iZXIgb2YgZmFybWVycyB1bnN1cnZleWVkIGluIHRoZSBmaXJzdCBzdXJ2ZXkgcm91bmQuIFNlZSBbdGhpcyBjbGVhbmluZyBmaWxlXSh3d3cuZ2l0aHViLmNvbSkgZm9yIG1vcmUgaW5mb3JtYXRpb24gb24gdGhlIGZhcm1lcnMgd2UgZGlkIG5vdCBmaW5kIGFnYWluIGluIHRoZSBmaXJzdCBmb2xsb3cgdXAuCgpIZXJlIEknbSBnb2luZyB0byBjYWxsIHRoZSBTVEFUQSBjbGVhbmluZyBmaWxlIHRvIG1ha2UgQVYncyBjaGFuZ2VzIHRvIHRoZSBSMSBmb2xsb3cgdXAgZGF0YS4gVGhpcyByZXF1aXJlcyB0aGF0IHRoZSBkYXRhIGZyb20gQ0MgaGF2ZSB0aGUgc2FtZSB2YXJpYWJsZSBuYW1lcyBhcyB0aGUgU1RBVEEgY2xlYW5pbmcgZmlsZS4gSSdtIGdvaW5nIHRvIHRyeSB0byBleGVjdXRlIHRoYXQgaGVyZToKCmBgYHtyfQpzdGF0YURpciA8LSBub3JtYWxpemVQYXRoKGZpbGUucGF0aCgiLi4iLCAicndfcm91bmRfMV9jaGVjayIpKQpgYGAKCkhlcmUgSSBhY2Nlc3MgdGhlIHNvaWwgcHJlZGljdGlvbnMgZnJvbSB0aGUgT0FGIHNvaWwgbGFiLiBbUGF0cmljayBCZWxsXShtYWlsdG86cGF0cmljay5iZWxsQG9uZWFjcmVmdW5kLm9yZykgbWFuYWdlcyB0aGUgbGFiIGFuZCBbTWlrZSBCYXJiZXJdKG1pa2UuYmFyYmVyQG9uZWFjcmVmdW5kLm9yZykgb3ZlcnNlZXMgdGhlIHByZWRpY3Rpb24gc2NyaXB0cy4KCmBgYHtyfQpzb2lsRGlyIDwtIG5vcm1hbGl6ZVBhdGgoZmlsZS5wYXRoKCIuLiIsICIuLiIsICJPQUYgU29pbCBMYWIgRm9sZGVyIiwgIlByb2plY3RzIiwgInJ3X3Noc19zZWNvbmRfcm91bmQiLCAiNF9wcmVkaWN0ZWQiLCAib3RoZXJfc3VtbWFyaWVzIikpCnNvaWwgPC0gcmVhZC5jc3YoZmlsZT1wYXN0ZShzb2lsRGlyLCAiY29tYmluZWQtcHJlZGljdGlvbnMtaW5jbHVkaW5nLWJhZC1vbmVzLmNzdiIsIHNlcCA9ICIvIikpCmBgYAoKIyMgQ29tYmluZSBiYXNlbGluZSBhbmQgUjEKCkNvbWJpbmUgdGhlIGF2YWlsYWJsZSBkYXRhIGJ5IGZhcm1lciBhbmQgcmVzb2x2ZSBtZXJnaW5nIGlzc3Vlcy4gVGhlc2UgZGF0YSBjYW4gYmUgY29tYmluZWQgbG9uZyBhcyBsb25nIGFzIHRoZSB2YXJpYWJsZSBuYW1lcyBhcmUgY29uc2lzdGVudCBvciB3aWRlLiBJJ20gZ29pbmcgdG8gY29tYmluZSB0aGUgZGF0YSBsb25nIGFuZCB1c2UgYHNwbGl0YCB0eXBlIGNvbW1hbmRzIHRvIGFnZ3JlZ2F0ZSB0aGUgZGF0YSBtb3JlIGVhc2lseS4gQ29uZmlybSB0aGUgdmFyaWFibGUgbmFtZXMgYXJlIGNvbnNpc3RlbnQuIEJ5IGFkdmFuY2luZyB0aGlzIGNvZGUgb24gNS85LzE3LCBJJ20gZm9yIHRoZSB0aW1lIGJlaW5nIGlnbm9yaW5nIHRoZSBjbGVhbmluZyBBbGV4IGRpZCBpbiBoaXMgZG8gZmlsZS4gSSdsbCBuZWVkIHRvIGdvIGJhY2sgYW5kIGluY29ycG9yYXRlIHRob3NlIGNoYW5nZXMuCgoqKlRPRE8qKjogc2VlIGlmIHRoZSB2YXJpYWJsZXMgbmFtZXMgaW4gQWxleCdzIHJhdyBkYXRhLCBzaGFyZWQgYnkgW05hdGhhbmllbF0obWFpbHRvOm5hdGhhbmllbC5yb3NlbmJsdW1Ab25lYWNyZWZ1bmQub3JnKSwgbWF0Y2ggdGhlIGRhdGEgSSdtIGRvd25sb2FkaW5nIGZyb20gY29tbWNhcmUuIElmIHNvLCBkb24ndCB1c2UgdGhlIGB2YXJfbmFtZXMueGxzeGAgc2hlZXQgYW5kIGluc3RlYWQgdXNlIHRob3NlIHZhcmlhYmxlIG5hbWVzIGFuZCBBbGV4J3MgZG8gZmlsZSB0byBwcmVzZXJ2ZSBhbGwgb2YgaGlzIGNoYW5nZXMuCgpOb3QgbWFueSBvZiB0aGUgbmFtZXMgYXJlIHRoZSBzYW1lLiBJJ3ZlIGRvd25sb2FkZWQgdGhlIG1ldGEgZGF0YSBmcm9tIENvbW1DYXJlIHdoaWNoIEknbGwgdXNlIHRvIHNpbXBsaWZ5IHRoZSBjbGVhbmluZyBvZiB0aGUgcm91bmQgMSBkYXRhLiBJJ20gYWxzbyBnb2luZyB0byByZXNoYXBlIHRoZSBiYXNlbGluZSB2YXJpYWJsZSBuYW1lcyB0byBzaW1wbGlmeSB0aGUgbWF0Y2hpbmcgb2YgYmFzZWxpbmUgdmFyaWFibGVzIHRvIHJvdW5kIDEgdmFyaWFibGVzLgpgYGB7ciwgbWVzc2FnZXM9Rn0KZGF0TmFtZXMgPC0gZnVuY3Rpb24oZGF0KXsKICB2YXJOYW1lcyA9IG5hbWVzKGRhdCkKICBleFZhbCA9IGRvLmNhbGwocmJpbmQsIGxhcHBseSh2YXJOYW1lcywgZnVuY3Rpb24oeCl7CiAgICB2YWwgPSBkYXRbMTozLHhdCiAgICByZXR1cm4odmFsKQogIH0pKQogIAogIG91dCA9IGNiaW5kKHZhck5hbWVzLCBleFZhbCkKICByZXR1cm4ob3V0KQp9CgpiYXNlTmFtZXMgPC0gZGF0TmFtZXMoYikKd3JpdGUuY3N2KGJhc2VOYW1lcywgZmlsZT0iYmFzZWxpbmUgdmFyIG5hbWVzLmNzdiIsIHJvdy5uYW1lcyA9IEYpCmBgYAoKTG9hZCBBbGV4J3MgcmF3IGRhdGEgYW5kIHRha2UgdGhlIHZhcmlhYmxlIG5hbWVzIGZyb20gdGhpcy4gSWYgSSBjYW4gYWxpZ24gdGhlc2UgdmFyaWFibGUgbmFtZXMgd2l0aCB0aGUgZGF0YSBmcm9tIENDIEkgY2FuIHRoZW4gZXhlY3V0ZSBBbGV4J3MgY2xlYW5pbmcgc2NyaXB0IG9uIHRoZSBDQyBkYXRhIGFuZCBwcm9jZWVkIHdpdGggY29tYmluaW5nIHRoZSBkYXRhCgojIyBTdGF0YSAuZG8gZmlsZQoKYGBge3J9CnJhd0RpciA8LSBub3JtYWxpemVQYXRoKGZpbGUucGF0aCgiU29pbCBoZWFsdGggc3R1ZHkgKHllYXIgb25lKSIsICJkYXRhIikpCgphdlJhdyA8LSByZWFkLmNzdihwYXN0ZShyYXdEaXIsICJ5MV9zaHNfcndhbmRhXzI4c2VwLmNzdiIsIHNlcCA9ICIvIiksIHN0cmluZ3NBc0ZhY3RvcnMgPSBGKQoKYGBgCgpJdCBsb29rcyBsaWtlIHRoZSBkYXRhIGZyb20gQ29tbUNhcmUgYWxpZ25zIHdpdGggdGhlIHJhdyBkYXRhIEFsZXggd29ya2VkIHdpdGggYXQgYGluZm9fZm9ybWlkYCB3aGljaCBpcyB0aGUgc2Vjb25kIGluZGV4IGZvciBgYXZSYXdgIGFuZCB0aGUgMTB0aCBpbmRleCBmb3IgYHJgLiBMZXQncyBqdXN0IHRyeSB0cmFuc2ZlcnJpbmcgdGhlbSBvdmVyIGFuZCB0aGUgd29yayBvZiB1cGRhdGluZyB0aGUgdmFyaWFibGUgbmFtZXMgdGhyb3VnaCB0aGUgQ0MgY29kZWJvb2sgZXhwb3J0IG1heSBub3QgYmUgbmVjZXNzYXJ5IQoKYGBge3J9CnZhclRlc3QgPC0gZGF0YS5mcmFtZShmcm9tY2MgPSBuYW1lcyhyKVsxMDo0MDldLCBmcm9tYXYgPSBuYW1lcyhhdlJhdylbMjo0MDFdKQojIGhlYWQodmFyVGVzdCkKIyB0YWlsKHZhclRlc3QpCiN2YXJUZXN0WzkwOjEyMCxdCmBgYAoKSXQgc2VlbXMgdG8gbGluZSB1cCBva2F5ICh3aXRoIHNvbWUgYWRqdXN0bWVudHMpISBUbyBpbmNvcnBvcmF0ZSBBbGV4J3MgY2xlYW5pbmcgY29kZSBJIGhhdmUgdG8gZXhwb3J0IHRoZSBkYXRhIGZyb20gUiB0byBhIGZvcm0gU3RhdGEgYWNjZXB0LCBydW4gdGhlIGNvZGUsIGFuZCB0aGVuIGxvYWQgdGhlIGRhdGEgYmFjayBpbi4KClRoaXMgZnVuY3Rpb24gd2lsbCByZW1vdmUgYWxsIHN0cmFuZ2Ugb3V0cHV0cyBmcm9tIHRoZSBkYXRhIGZyb20gQ29tbUNhcmUgc28gdGhhdCB0aGUgU1RBVEEgY29kZSB3b3JrcwoKYGBge3J9CmNoYXJDbGVhbiA8LSBmdW5jdGlvbihkZil7CiAgCiAgZGYgPC0gYXMuZGF0YS5mcmFtZShsYXBwbHkoZGYsIGZ1bmN0aW9uKHgpewogIHggPSBnc3ViKCInIiwgJycsIHgpCiAgeCA9IGdzdWIoIl5iIiwgJycsIHgpCiAgeCA9IGlmZWxzZShncmVwbCgibWFwIG9iamVjdCIsIHgpPT1ULCBOQSwgeCkKICByZXR1cm4oeCkKICB9KSkKcmV0dXJuKGRmKQp9CgpyIDwtIGNoYXJDbGVhbihyKQpgYGAKCmBgYHtyfQpuYW1lcyhyKVsxMDo0MDldIDwtIG5hbWVzKGF2UmF3KVsyOjQwMV0KCiNleHBvcnQgc28gc3RhdGEgY2FuIHJ1biAtIGNoZWNrIGZvciB2YXJpYWJsZSBuYW1lcyBsb25nZXIgdGhhbiAzMmNoYXIKdGFibGUobmNoYXIobmFtZXMocikpKQoKd3JpdGUuY3N2KHIsIGZpbGU9InRvQmVDbGVhbmVkU3RhdGEuY3N2Iiwgcm93Lm5hbWVzID0gRikKCnN0YXRhKCJjbGVhbnNfeTFfc2hzX3J3YW5kYS5kbyIsIHN0YXRhLmVjaG89RikKYGBgCgpOb3cgbG9hZCB0aGUgcmVzdWx0IG9mIHRoZSBTdGF0YSBmaWxlCmBgYHtyfQpyIDwtIHJlYWQuY3N2KCJjbGVhbmVkZm9yUi5jc3YiLCBzdHJpbmdzQXNGYWN0b3JzID0gRikKYGBgCgpgYGB7cixldmFsPUZ9Cm5ld05hbWUgPC0gcmVhZF9leGNlbCgidmFyX25hbWVzLnhsc3giLCBzaGVldD0xKQoKcVR5cGVzIDwtIGMoIk11bHRpcGxlIENob2ljZSIsICJQaG9uZSBOdW1iZXIgb3IgTnVtZXJpYyBJRCIsICJDaGVja2JveCIsICJUZXh0IiwgIkRlY2ltYWwiLCAiSW1hZ2UgQ2FwdHVyZSIsICJCYXJjb2RlIFNjYW4iLCAiR1BTIiwgIkludGVnZXIiLCAiVGltZSIsICJEYXRlIikKCm5ld05hbWUgPC0gbmV3TmFtZSAlPiUgZHBseXI6OnNlbGVjdCgxOjYKKSAlPiUgZHBseXI6OmZpbHRlcihuZXdOYW1lJFR5cGUgJWluJSBxVHlwZXMpICU+JSBhcy5kYXRhLmZyYW1lKCkKCiNuZXdOYW1lIDwtIG5ld05hbWUgJT4lIGZpbHRlcihuZXcudmFyLm5hbWUhPSJnZW5lcmFsLmNvbW1lbnQiKQptZXRhVmFycyA8LSBuYW1lcyhyKVsxOjEwXQpuZXdOYW1lVmFycyA8LSBjKG1ldGFWYXJzLCBuZXdOYW1lJG5ldy52YXIubmFtZSkKCmxlbmd0aChuZXdOYW1lVmFycyk9PWRpbShyKVsyXQoKd3JpdGUuY3N2KG5ld05hbWVWYXJzLCBmaWxlPSJuZXdWYXIgY2hlY2suY3N2IikKd3JpdGUuY3N2KG5hbWVzKHIpLCBmaWxlPSJyb3VuZDEgVmFyIGNoZWNrLmNzdiIpCgpgYGAKCmBgYHtyLCBldmFsPUZ9Cm5hbWVzKHIpIDwtIG5ld05hbWVWYXJzCgojIGRyb3AgdmFycyB3aXRoIGRyb3AKciA8LSByWywtd2hpY2goZ3JlcGwoImRyb3AuIiwgbmFtZXMocikpKV0KYGBgCgoKYGBge3IsIGV2YWw9Rn0KcU51bSA8LSBjKCJQaG9uZSBOdW1iZXIgb3IgTnVtZXJpYyBJRCIsICJEZWNpbWFsIiwgIkludGVnZXIiKQpudW1zIDwtIG5ld05hbWVbbmV3TmFtZSRUeXBlICVpbiUgcU51bSwgIm5ldy52YXIubmFtZSJdCm51bXMgPC0gbnVtc1std2hpY2goZ3JlcGwoImRyb3AuIiwgbnVtcykpXQoKdG9SZW1vdmUgPC0gYygicGhvbmUiLCAib2FmaWQiKQpudW1zIDwtIG51bXNbIW51bXMgJWluJSB0b1JlbW92ZV0KCiMgYWRkIGluIHBsb3Quc2l6ZQojbnVtcyA8LSBjKG51bXMsICJwbG90LnNpemUiKQoKclssIG51bXNdIDwtIGFzLmRhdGEuZnJhbWUobGFwcGx5KHJbLG51bXNdLCBmdW5jdGlvbih4KXsKICBhcy5udW1lcmljKGFzLmNoYXJhY3Rlcih4KSkKfSkpCmBgYAoKIyBDbGVhbmluZwoKVGhlIGByYCBkYXRhZnJhbWUgaGFzIG1hbnkgbW9yZSB2YXJpYWJsZXMgdGhhbiB0aGUgYmFzZWxpbmUgc3VydmV5LiBUaGlzIHdhcyBpbiBwYXJ0IGV4cGVjdGVkOyB3ZSBhZGRlZCBxdWVzdGlvbnMgdG8gdGhlIGZpcnN0IGZvbGxvdyB1cCByb3VuZCBiYXNlZCBvbiBsZXNzb25zIGZyb20gdGhlIGJhc2VsaW5lLiBJdCdzIGFsc28gZHVlIHRvIGhvdyB0aGUgc3VydmV5IHdhcyBzZXQgdXAgaW4gQ29tbUNhcmUuIEJlZm9yZSBjb21iaW5pbmcgdGhlIGJhc2VsaW5lIGFuZCB0aGUgZmlyc3QgZm9sbG93IHVwIHJvdW5kIEkgbmVlZCB0bzoKCiogcmVzaGFwZSB0aGUgcm91bmQgMSB2YXJpYWJsZXMgc28gdGhhdCB0aGV5IGFwcHJvcHJpYXRlbHkgbWF0Y2ggdGhlIGJhc2VsaW5lIHZhcmlhYmxlcwoqIENsZWFuIHRob3NlIHZhcmlhbGVzIG9yIHByZXBhcmUgdGhlbSBhcyBuZWVkIGJlIGZvciBhIAoqIEZvciB2YXJpYWJsZXMgd2l0aCBubyBtYXRjaCwgY2xlYW4KCmBgYHtyfQptdWx0aXBsb3QgPC0gZnVuY3Rpb24oLi4uLCBwbG90bGlzdD1OVUxMLCBmaWxlLCBjb2xzPTEsIGxheW91dD1OVUxMKSB7CiAgbGlicmFyeShncmlkKQoKICAjIE1ha2UgYSBsaXN0IGZyb20gdGhlIC4uLiBhcmd1bWVudHMgYW5kIHBsb3RsaXN0CiAgcGxvdHMgPC0gYyhsaXN0KC4uLiksIHBsb3RsaXN0KQoKICBudW1QbG90cyA9IGxlbmd0aChwbG90cykKCiAgIyBJZiBsYXlvdXQgaXMgTlVMTCwgdGhlbiB1c2UgJ2NvbHMnIHRvIGRldGVybWluZSBsYXlvdXQKICBpZiAoaXMubnVsbChsYXlvdXQpKSB7CiAgICAjIE1ha2UgdGhlIHBhbmVsCiAgICAjIG5jb2w6IE51bWJlciBvZiBjb2x1bW5zIG9mIHBsb3RzCiAgICAjIG5yb3c6IE51bWJlciBvZiByb3dzIG5lZWRlZCwgY2FsY3VsYXRlZCBmcm9tICMgb2YgY29scwogICAgbGF5b3V0IDwtIG1hdHJpeChzZXEoMSwgY29scyAqIGNlaWxpbmcobnVtUGxvdHMvY29scykpLAogICAgICAgICAgICAgICAgICAgIG5jb2wgPSBjb2xzLCBucm93ID0gY2VpbGluZyhudW1QbG90cy9jb2xzKSkKICB9CgogaWYgKG51bVBsb3RzPT0xKSB7CiAgICBwcmludChwbG90c1tbMV1dKQoKICB9IGVsc2UgewogICAgIyBTZXQgdXAgdGhlIHBhZ2UKICAgIGdyaWQubmV3cGFnZSgpCiAgICBwdXNoVmlld3BvcnQodmlld3BvcnQobGF5b3V0ID0gZ3JpZC5sYXlvdXQobnJvdyhsYXlvdXQpLCBuY29sKGxheW91dCkpKSkKCiAgICAjIE1ha2UgZWFjaCBwbG90LCBpbiB0aGUgY29ycmVjdCBsb2NhdGlvbgogICAgZm9yIChpIGluIDE6bnVtUGxvdHMpIHsKICAgICAgIyBHZXQgdGhlIGksaiBtYXRyaXggcG9zaXRpb25zIG9mIHRoZSByZWdpb25zIHRoYXQgY29udGFpbiB0aGlzIHN1YnBsb3QKICAgICAgbWF0Y2hpZHggPC0gYXMuZGF0YS5mcmFtZSh3aGljaChsYXlvdXQgPT0gaSwgYXJyLmluZCA9IFRSVUUpKQoKICAgICAgcHJpbnQocGxvdHNbW2ldXSwgdnAgPSB2aWV3cG9ydChsYXlvdXQucG9zLnJvdyA9IG1hdGNoaWR4JHJvdywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsYXlvdXQucG9zLmNvbCA9IG1hdGNoaWR4JGNvbCkpCiAgICB9CiAgfQp9CmBgYAoKIyMgRHJvcCB2YXJpYWJsZXMKYGBge3J9CnRvRHJvcCA8LSBjKCJhcHBmb3JtaWQiLCAiaWQiLCAiZG9tYWluIiwgIm1ldGFkYXRhZGV2aWNlaWQiKQpyIDwtIHJbLCFuYW1lcyhyKSAlaW4lIHRvRHJvcF0KYGBgCgoKYGBge3J9CnNvdXJjZSgiLi4vb2FmbGliL21pc2MuUiIpCm5hbWVzKHIpIDwtIGdzdWIoIl55MV98aW50cm9fIiwgIiIsIG5hbWVzKHIpKQpyW3I9PSIuIl0gPC0gTkEKCnIgPC0gZGl2aWRlR3BzKHIsICJncHNfY29vcmQiKQpgYGAKCiMjIENhdGVnb3JpY2FsIHZhcmlhYmxlcwoKVGhlIHJlc3BvbnNlcyBvZiB0aGUgY2F0ZWdvcmljYWwgdmFyaWFibGVzIHNob3VsZCBiZSByZWd1bGF0ZWQgdGhyb3VnaCBDQywgaG93ZXZlciwgdG8gY2hlY2ssIG1ha2UgYSB0YWJsZSB0aGF0IHNob3dzIHRoZSB0b3AgdGVuIHJlc3BvbnNlcyBpbiBkZXNjZW5kaW5nIG9yZGVyIGFuZCBtYWtlIGEgZ3JhcGggb2YgcmVzcG9uc2UgY291bnRzIHRvIGtub3cgd2hhdCB0byBjaGVjay4gSSdsbCB0aGVuIGNhcHR1cmUgYW55IGNoYXJhY3RlcnMgdGhhdCBzaG91bGQgYmUgbnVtZXJpYyBhbmQgY29udmVydCB0aGVtLgoKYGBge3J9CmNhdFZhcnMgPC0gbmFtZXMocilbc2FwcGx5KHIsIGZ1bmN0aW9uKHgpewogIGlzLmNoYXJhY3Rlcih4KQp9KV0KCmVudW1DbGVhbiA8LSBmdW5jdGlvbihkYXQsIHgsIHRvUmVtb3ZlKXsKICBkYXRbLHhdIDwtIGlmZWxzZShkYXRbLHhdICVpbiUgdG9SZW1vdmUsIE5BLCBkYXRbLHhdKQogIHJldHVybihkYXRbLHhdKQp9CgpzdHJUYWJsZSA8LSBmdW5jdGlvbihkYXQsIHgpewogIHZhck5hbWUgPSB4CiAgdGFiID0gYXMuZGF0YS5mcmFtZSh0YWJsZShkYXRbLHhdLCB1c2VOQSA9ICdpZmFueScpKQogIHRhYiA9IHRhYltvcmRlcih0YWIkRnJlcSwgZGVjcmVhc2luZyA9IFQpLF0KICBlbmQgPSBpZmVsc2UobGVuZ3RoKHRhYiRWYXIxKTwxMCwgbGVuZ3RoKHRhYiRWYXIxKSwgMTApCiAgcmVwT3JkZXIgPSBwYXN0ZSh0YWIkVmFyMVsxOmVuZF0sIGNvbGxhcHNlPSIsICIpCiAgb3V0ID0gZGF0YS5mcmFtZSh2YXJpYWJsZSA9IHZhck5hbWUsCiAgICAgICAgICAgICAgICAgICByZXNwb25zZXMgPSByZXBPcmRlcikKICAKICByZXR1cm4ob3V0KQp9CgojIGNsZWFuIHVwIGtub3duIHZhbHVlcwpjYXRFbnVtVmFscyA8LSBjKCItOTkiLCAiLTg4IiwgIi0gOTkiLCAiLTk5LjAiLCAiODgiLCAiXzg4IiwgIi0gODgiLCAiMC44OCIsCiAgICAgICAgICAgICAgICAgIi0tODgiLCAiX184OCIsICItODguMCIsICI5OS4wIikKclssY2F0VmFyc10gPC0gc2FwcGx5KGNhdFZhcnMsIGZ1bmN0aW9uKHkpewogIHJbLHldIDwtIGVudW1DbGVhbihyLHksIGNhdEVudW1WYWxzKQp9KQoKCnJlc3BvbnNlVGFibGUgPC0gZG8uY2FsbChyYmluZCwgbGFwcGx5KGNhdFZhcnMsIGZ1bmN0aW9uKHgpewogIHN0clRhYmxlKHIsIHgpCn0pKQoKYGBgCgojIyMgQ2F0ZWdvcmljYWwgcmVzcG9uc2UgdGFibGUKCkEgc2ltcGxlIHRhYmxlIHRvIHByZXZpZXcgdGhlIHZhbHVlcyBpbiB0aGUgZGF0YS4gVGhlIHZhbHVlcyBhcmUgcmFua2VkIGJ5IGZyZXF1ZW5jeS4KCmBgYHtyfQprYWJsZShyZXNwb25zZVRhYmxlKQpgYGAKCiMjIyBDYXRlZ29yaWNhbCByZXNwb25zZSBncmFwaHMKYGBge3J9CnJlcEdyYXBocyA8LSBmdW5jdGlvbihkYXQsIHgpewogIHRhYiA9IGFzLmRhdGEuZnJhbWUodGFibGUoZGF0Wyx4XSwgdXNlTkEgPSAnaWZhbnknKSkKICB0YWIgPSB0YWJbb3JkZXIodGFiJEZyZXEsIGRlY3JlYXNpbmcgPSBUKSxdCiAgcHJpbnQoCiAgICBnZ3Bsb3QoZGF0YT10YWIsIGFlcyh4PVZhcjEsIHk9RnJlcSkpICsgZ2VvbV9iYXIoc3RhdD0iaWRlbnRpdHkiKSArCiAgICAgIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iLCBheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSArCiAgICAgIGxhYnModGl0bGUgPXBhc3RlMCgiQ29tcG9zaXRpb24gb2YgdmFyaWFibGU6ICIsIHgpKQogICkKfQoKYWRtaW5WYXJzIDwtIGMobmFtZXMocilbZ3JlcCgibWV0YSIsIG5hbWVzKHIpKV0sICJzdGFydF90aW1lIiwgImVudW1fbmFtZSIsICJwaG90byIsICJjZWxsX2ZpZWxkIiwgInZpbGxhZ2UiLCAiZmFybWVyX3Jlc3BvbmQiLCAiZmFybWVyX3Bob25lbnVtYmVyIiwgImRfcGhvbmUiLCAibmVpZ2hib3JfcGhvbmVudW1iZXIiLCAiZmFybWVyX2xpc3QiLCAidW5pcXVlX2xvY2F0aW9uIiwgImNvbW1lbnRzIiwgImdwc19jb29yZCIsICJzYW1wbGVfaWQiKQpub25BZG1pblZhcnMgPC0gY2F0VmFyc1shY2F0VmFycyAlaW4lIGFkbWluVmFyc10KCmZvcihpIGluIDE6bGVuZ3RoKG5vbkFkbWluVmFycykpewogIHJlcEdyYXBocyhyLCBub25BZG1pblZhcnNbaV0pCn0KYGBgCgojIyMgTWFudWFsIGNoYXJhY3RlciBjbGVhbmluZwpgYGB7cn0KciRmZW1hbGUgPC0gaWZlbHNlKHIkZ2VuZGVyPT0iZmVtYWxlIiwgMSwgMCkKciRkaXN0cmljdCA8LSBpZmVsc2UoZ3JlcGwoIm55YW56YSIsIHIkZGlzdHJpY3QpPT1ULCAiTnlhbnphIiwgciRkaXN0cmljdCkKCnRhYmxlKHIka2dfc2VlZF8xNmJfMSkKdGFibGUociRrZ195aWVsZF8xNmFfMikKCnN0cnRvTnVtIDwtIGMoImtnX3NlZWRfMTZiXzEiLCAia2dfeWllbGRfMTZhXzEiLCAia2dfeWllbGRfMTZiXzEiLCAia2dfeWllbGRfMTZiXzIiKQpyWyxzdHJ0b051bV0gPC0gc2FwcGx5KHJbLHN0cnRvTnVtXSwgZnVuY3Rpb24oeCl7YXMubnVtZXJpYyh4KX0pCmBgYAoKTm90ZXMgb24gdGhlIGNhdGVnb3JpY2FsIHZhcmlhYmxlczoKCiogV2UgZG9uJ3QgaGF2ZSBtYW55IGFjdHVhbCByZXNwb25zZXMgb24gc2VlZCB0eXBlIGRlc3BpdGUgYWxsIGZhcm1lcnMgdGVsbGluZyB1cyBhYm91dCBhIGNyb3AgdGhleSBhcmUgZ3Jvd2luZy4gV2h5PyBDaGVjayB0aGF0IHRoZXJlIHdhc24ndCBhIG1pc2xhYmVsaW5nIG9mIHZhcmlhYmxlcy4KKiBDaGVjayB0aGUgJ3doaWNoX21haXplX3NlZWQnIHZhcmlhYmxlcyB0byBtYWtlIGNlcnRhaW4gdGhleSdyZSBmbGV4aWJsZSB0byB0aGUgdHlwZSBvZiBjcm9wIHNlbGVjdGVkIGluIHRoZSBwcmV2aW91cyBxdWVzdGlvbi4KKiBDb25maXJtIHRoYXQgYmxhbmsgaXMgTkEgbm90IDAuCgojIyBOdW1lcmljIHZhcmlhYmxlcwoKYGBge3J9Cm51bVZhcnMgPC0gbmFtZXMocilbc2FwcGx5KHIsIGZ1bmN0aW9uKHgpewogIGlzLm51bWVyaWMoeCkKfSldCmBgYAoKQmFzaWMgY2xlYW5pbmcgb2Yga25vd24gaXNzdWVzIGxpa2UgZW51bWVyYXRvciBjb2RlcyBmb3IgREssIE5XUiwgZXRjLgpgYGB7cn0KZW51bVZhbHMgPC0gYygtODgsLTg1LCAtOTkpCgpyWyxudW1WYXJzXSA8LSBzYXBwbHkobnVtVmFycywgZnVuY3Rpb24oeSl7CiAgclsseV0gPC0gZW51bUNsZWFuKHIseSwgZW51bVZhbHMpCn0pCmBgYAoKIyMjIE51bWVyaWMgb3V0bGllciB0YWJsZQoKYGBge3J9Cmlxci5jaGVjayA8LSBmdW5jdGlvbihkYXQsIHgpIHsgCiAgcTEgPSBzdW1tYXJ5KGRhdFsseF0pW1syXV0KICBxMyA9IHN1bW1hcnkoZGF0Wyx4XSlbWzVdXSAKICBpcXIgPSBxMy1xMQogIG1hcmsgID0gaWZlbHNlKGRhdFsseF0gPCAocTEgLSAoMS41KmlxcikpIHwgZGF0Wyx4XSA+IChxMyArICgxLjUqaXFyKSksIDEsMCkKICB0YWIgPSByYmluZCgKICAgIHN1bW1hcnkoZGF0Wyx4XSksCiAgICBzdW1tYXJ5KGRhdFttYXJrPT0wLCB4XSkKICApCiAgcmV0dXJuKHRhYikKfQoKIyByZW1vdmUgYWRtaW4gdmFycwpudW1BZG1pblZhcnMgPC0gYyhudW1WYXJzWzE6M10pCm51bVZhcnNOb3RBZG1pbiA8LSBudW1WYXJzWyFudW1WYXJzICVpbiUgbnVtQWRtaW5WYXJzXQoKaXFyVGFiIDwtIGRvLmNhbGwocGx5cjo6cmJpbmQuZmlsbCwgbGFwcGx5KG51bVZhcnNOb3RBZG1pbiwgZnVuY3Rpb24oeSl7CiAgI3ByaW50KHkpCiAgcmVzID0gaXFyLmNoZWNrKHIsIHkpCiAgI3ByaW50KGRpbShyZXMpKQogIG91dCA9IGRhdGEuZnJhbWUodmFyPXJiaW5kKHksIHBhc3RlKHksICIuaXFyIiwgc2VwPSIiKSksIHJlcykKICByZXR1cm4ob3V0KQp9KSkKCmlxclRhYlssMjo4XSA8LSBzYXBwbHkoaXFyVGFiWywyOjhdLCBmdW5jdGlvbih4KXtyb3VuZCh4LDEpfSkKYGBgCgpUaGUgb3V0bGllciB0YWJsZSBzdW1tYXJpemVzIHRoZSBudW1lcmljIHZhcmlhYmxlcyB3aXRoIGFuZCB3aXRob3V0IElRUiBvdXRsaWVycyB0byBzaG93IGhvdyB0aGUgZGF0YSBjaGFuZ2VzIGJhc2VkIG9uIHRoaXMgZmlsdGVyLgoKYGBge3J9CmtuaXRyOjprYWJsZShpcXJUYWIsIHJvdy5uYW1lcyA9IEYsIGRpZ2l0cyA9IDAsIGZvcm1hdCA9ICdodG1sJykKYGBgCgojIyMgT3V0bGllciBHcmFwaHMKYGBge3J9CiMgaHR0cDovL3Jmb3JwdWJsaWNoZWFsdGguYmxvZ3Nwb3QuY29tLzIwMTQvMDIvZ2dwbG90Mi1jaGVhdHNoZWV0LWZvci12aXN1YWxpemluZy5odG1sCmZvcihpIGluIDE6bGVuZ3RoKG51bVZhcnNOb3RBZG1pbikpewogICAgYmFzZSA8LSBnZ3Bsb3QociwgYWVzKHg9clssbnVtVmFyc05vdEFkbWluW2ldXSkpICsgbGFicyh4ID0gbnVtVmFyc05vdEFkbWluW2ldKQogICAgdGVtcDEgPC0gYmFzZSArIGdlb21fZGVuc2l0eSgpCiAgICB0ZW1wMiA8LSBiYXNlICsgZ2VvbV9oaXN0b2dyYW0oKQogICAgI3RlbXAyIDwtIGJveHBsb3QoclssbnVtVmFyc1tpXV0sbWFpbj1wYXN0ZTAoIlZhcmlhYmxlOiAiLCBudW1WYXJzW2ldKSkKICAgIG11bHRpcGxvdCh0ZW1wMSwgdGVtcDIsIGNvbHMgPSAyKQp9CmBgYAoKIyMgQ2hlY2sgZm9yIHVuaXF1ZSBpZHMKCkknbSBzZWVpbmcgdGhhdCB0aGVyZSBhcmUgZHVwbGljYXRlZCBmYXJtZXJzIGluIHRoZSBkYXRhIHdoZW4gSSdtIHRyeWluZyB0byByZXNoYXBlIHRoZSBgcmAgZGF0YSBmcm9tIHdpZGUgdG8gbG9uZy4gTGV0J3MgY2hlY2sgdGhlbSBvdXQgaGVyZSBhbmQgc2VlIGlmIHdlIGNhbiBmaWd1cmUgb3V0IHdoaWNoIG9ic2VydmF0aW9uIGlzIHJpZ2h0LiAKCiogQ2hlY2sgQWxleCdzIGRvIGZpbGUgdG8gc2VlIGlmIHRoZXJlJ3MgbWVudGlvbiBvZiB0aGVzZSBmYXJtZXJzLiBbTm8gbWVudGlvbl0KKiBDaGVjayB0aGUgYmFzZWxpbmUgdmFsdWVzIGFzIHRoZXNlIHNob3VsZCBsaW5lIHVwLgoKYGBge3J9Cmxlbmd0aChyJHNhbXBsZV9pZCk9PWxlbmd0aCh1bmlxdWUociRzYW1wbGVfaWQpKQpkdXBzIDwtIHIkc2FtcGxlX2lkW2R1cGxpY2F0ZWQociRzYW1wbGVfaWQpXQpkdXBJbmRleCA8LSB3aGljaChkdXBsaWNhdGVkKHIkc2FtcGxlX2lkKSkKCiNkdXBEYXQgPC0gcltyJHNhbXBsZV9pZCAlaW4lIGR1cHMsXQpoZWFkKHJbciRzYW1wbGVfaWQ9PWR1cHNbMV0sXSkKaGVhZChyW3Ikc2FtcGxlX2lkPT1kdXBzWzJdLF0pCmBgYAoKTGV0J3Mgc29sdmUgdGhlIHVuaXF1ZSBpZCBpc3N1ZSBieSBsb29raW5nIGF0IGlkZW50aWZ5aW5nIGluZm9ybWF0aW9uIGluIHRoZSBiYXNlbGluZSBkYXRhCmBgYHtyfQoKCnJvdW5kSWQgPC0gciAlPiUKICBkcGx5cjo6c2VsZWN0KGRpc3RyaWN0LCBjZWxsX2ZpZWxkLCB2aWxsYWdlLCBzYW1wbGVfaWQsIGZhcm1lcl9saXN0KSAlPiUKICBmaWx0ZXIociRzYW1wbGVfaWQgJWluJSBkdXBzKQoKCgojZApsb2FkKCJyYXdCYXNlbGluZVdpdGhJZGVudGlmZXJzLlJkYXRhIikKYmFzZUlkIDwtIGQgJT4lIAogIGRwbHlyOjpzZWxlY3QoZGlzdHJpY3QsIHNlbGVjdGVkX2NlbGwsIHVtdWR1Z3VkdSwgIHNhbXBsZV9pZCwgZmFybWVyX25hbWUgKSAlPiUKICBmaWx0ZXIoZCRzYW1wbGVfaWQgJWluJSBkdXBzKQoKI2Jhc2VJZApyb3VuZElkCgpgYGAKCiMjIyBDb3JyZWN0IGR1cGxpY2F0ZXMKCkNvcnJlY3QgdGhlIGR1cGxpY2F0ZXMgSSBjYW4gYW5kIGRyb3AgdGhlIG90aGVycyBmb3Igbm93LiBGbGFnIHRoZSBkdXBsaWNhdGVkIG9uZXMgYW5kIHNhdmUgdGhlbSB0byBzaGFyZSB3aXRoIE5hdGhhbmllbC4KClRPRE8obWF0dGxvd2VzKSAtIHNoYXJlIGFueSByZW1haW5pbmcgZHVwbGljYXRlcyB3aXRoIE5hdGhhbmllbCBhbmQgc2VlIGlmIGhlIGhhcyBhIHNvbHV0aW9uLiBBbHNvIHNlZSBpZiBoZSBjYW4gdW5kZXJzdGFuZCB3aHkgdGhpcyBtaWdodCBoYXZlIGhhcHBlbmVkIGFuZCBpZiB0aGV5IHNob3VsZCBhY3R1YWxseSBoYXZlIGEgZGlmZmVyZW50IHNhbXBsZSBpZC4KCmBgYHtyfQpyIDwtIHIgJT4lIG11dGF0ZSgKICAgIGR1cCA9IGlmZWxzZSgKICAgICAgc2FtcGxlX2lkID09ICIxMiIgJiBjZWxsX2ZpZWxkID09ICJNVU5BTklSQSIgfAogICAgICBzYW1wbGVfaWQgPT0gIjEzNyIgJiB2aWxsYWdlID09ICJSdXN1bWEiIHwKICAgICAgc2FtcGxlX2lkID09ICIxNTAzIiAmIGZhcm1lcl9saXN0PT0iTkFLQUdJWkUgVmFsXFx4YzNcXHhhOXJpZSIgfAogICAgICAjc2FtcGxlX2lkID09ICIyMDQ0QyIgJiAgIyBzYW1lIQogICAgICBzYW1wbGVfaWQgPT0gIjIyNzgiICYgY2VsbF9maWVsZD09Ik5raXJhIEEiIHwgIyBjaGVjayB0aGlzIGFzIG1heWJlIHRoaXMgd2FzIHRoZSBvbmx5IHRoaW5nIHdyb25nPwogICAgICAjc2FtcGxlX2lkID09ICIyMjk5IiAmICMgc2FtZSEKICAgICAgc2FtcGxlX2lkID09ICIyNjEwIiAmIHZpbGxhZ2U9PSJhZ2FraXJpIiAjfCAgI2FnYWtpcmkgaXMgY2xvc2UgdG8gZ2FraXJpIGluIHNwZWxsaW5nLiBJcyB0aGlzIGp1c3QgYSB0eXBvPwogICAgICAjc2FtcGxlX2lkID09ICIyNjEyIiAmICAjIHNhbWUgbmFtZXMhCiAgICAgICNzYW1wbGVfaWQgPT0gIjI2MTJDIiAjIHNhbWUgbmFtZXMhCiAgICAgICwgMSwgMCkKKSAlPiUgZmlsdGVyKAogIGR1cCE9MQopICU+JSBkcGx5cjo6c2VsZWN0KC1kdXApIAoKIyBydW4gdGhpcyBjb2RlIGFnYWluIGZyb20gYWJvdmUgdG8gZ2V0IHVwZGF0ZWQgZHVwbGljYXRlcyBsaXN0CiNsZW5ndGgociRzYW1wbGVfaWQpPT1sZW5ndGgodW5pcXVlKHIkc2FtcGxlX2lkKSkKZHVwcyA8LSByJHNhbXBsZV9pZFtkdXBsaWNhdGVkKHIkc2FtcGxlX2lkKV0KZHVwSW5kZXggPC0gd2hpY2goZHVwbGljYXRlZChyJHNhbXBsZV9pZCkpCgojIGZvciB0aGUgdGltZSBiZWluZyBkcm9wIHRoZSBvYnNlcnZhdGlvbnMgdGhhdCBhcmUgZHVwbGljYXRlcwpyIDwtIHJbIXIkc2FtcGxlX2lkICVpbiUgZHVwcyxdCgpgYGAKCiMjIFJlc2hhcGUgdmFyaWFibGVzCgpUaGlzIHNob3VsZCBpbmNsdWRlIHRoZSBiYXNlbGluZSB2YXJpYWJsZXMgYXMgd2VsbC4KCkxldCdzIGZpcnN0IGNoZWNrIHdpdGggdGhlIGJhc2VsaW5lIGRhdGEgdG8gc2VlIHdoYXQgdmFyaWFibGVzIHdlIG1hZGUgdGhlcmUgc28gSSBjYW4gbWFrZSB0aGUgc2FtZSBvbmVzIGZyb20gdGhlIHJvdW5kIDEgZGF0YS4gVGhlcmUgYXJlIHNvbWUgdmFyaWFibGVzIHRoYXQgYXJlIGJhc2VsaW5lIHZhcmlhYmxlcyBvbmx5IGxpa2UgdmFyaWFibGVzIGFza2luZyBhYm91dCBoaXN0b3JpY2FsIHByYWN0aWNlcy4gVGhlcmUgYXJlIHRoZW4gb3RoZXIgdmFyaWFibGVzIHRoYXQgd2lsbCB2YXJ5IGJ5IHNlYXNvbi4gVGhlc2UgYXJlIHRoZSB2YXJpYWJsZXMgdGhhdCB3ZSB1bHRpbWF0ZWx5IHdhbnQgaW4gdG8gc2hhcGUgaW4gYSBsb25nIGRhdGFzZXQgYnkgc2Vhc29uIHRvIGFuYWx5emUgY2hhbmdlcyBvdmVydGltZSBpbiBwcmFjdGljZXMgYW5kIHNvaWwgbWFuYWdlbWVudC4gSSB0aGluayB0aGlzIHdpbGwgcmVzdWx0IGluIGEgZGF0YXNldCB0aGF0IGhhcyBvbmUgcm93IHBlciBmYXJtZXIgcGVyIHNlYXNvbi4gU29tZSB2YXJpYWJsZXMgbWF5IG5vdCBmaXQgbmljZWx5IGludG8gdGhpcyBidXQgd2UgY2FuIGRlYWwgd2l0aCB0aG9zZS4gRm9yIHZhcmlhYmxlcyB0aGF0IGFyZW4ndCBjaGFuZ2luZyBvdmVyIHRpbWUgdGhleSdsbCBzaG93IGFzIG5vdCBpbXBvcnRhbnQgaW4gb3VyIG1vZGVsLiBUaGV5J3JlIGltcG9ydGFudCBmb3IgbWF0Y2hpbmcgZmFybWVycy4KClRoZXJlIGFyZSBhIGxvdCBvZiB2YXJpYWJsZXMgdG8gdHJ5IHRvIGxpbmUgdXAuIFNvbWUgYWxyZWFkeSBoYXZlIHRoZSBzYW1lIG5hbWUgYnV0IGhvdyB0byBiZXN0IGNvbWJpbmUgdGhlIG9uZXMgdGhhdCBoYXZlIGRpZmZlcmVudCB2YXJpYWJsZSBuYW1lcz8gSSdtIGdvaW5nIHRvIHdyaXRlIGEgZnVuY3Rpb24gdGhhdCB0YWtlcyBhIHZhcmlhYmxlIG5hbWUgZnJvbSBgYmAgYW5kIGEgdmFyaWFibGUgbmFtZSBmcm9tIGByYCB0aGF0IHNob3VsZCBnbyB0b2dldGhlciwgdXBkYXRlcyB0aGUgYHJgIHZhcmlhYmxlIG5hbWUgYW5kIHVzZXMgdGhhdCBpbmZvIHRvIGByYmluZGAgdGhlIGRhdGEgaW50byBhIGxvbmcgZGF0YXNldC4KCmBgYHtyfQojIG5hbWVzKGIpCiMgbmFtZXMocikKCiMgY2hlY2sgdGhlIG5hbWVzIHRoYXQgYWxyZWFkeSBtYXRjaApiYXNlbGluZUZvdW5kIDwtIG5hbWVzKGIpW25hbWVzKGIpICVpbiUgbmFtZXMocildICMgbm90IG1hbnkgdmFyaWFibGUgbmFtZXMgYXJlIGFsaWduZWQKYGBgCgpVcGRhdGUgdmFyaWFibGUgbmFtZXMgc28gdGhhdCBhbnkgdmFyaWFibGUgd2l0aCAxNmEgb3IgMTZiIGhhcyBhIHRoZSBgYWAgb3IgYGJgIHNlYXNvbiBkZXNpZ25hdGlvbiBhdCB0aGUgZW5kIGl0IHNvIEkgY2FuIHJlcGxpY2F0ZSB0aGUgYGdhdGhlcigpYCBhbmQgYHNwcmVhZCgpYCBvcHRpb25zIGZvciByZW9yZ2FuaXppbmcgdGhlIGRhdGEgYnkgc2Vhc29uIGFuZCBieSBwbG90LiBUaGlzIG1lYW5zIHRoYXQgdGhlIHZhcmlhYmxlIG5hbWVzIHdpbGwgcmV0YWluIHRoZWlyIGRlc2lnbmF0aW9uIG9mIGZpcnN0IG9yIHNlY29uZCBhcHBsaWNhdGlvbiBhbmQgYmUgZGlzdGluZ3Vpc2hhYmxlLgoKVE9ETyhtYXR0bG93ZXMpIC0gcmVuYW1lIHRoZSB2YXJpYWJsZXMgYWNjb3JkaW5nIHRvIHRoYXQgY29udmVudGlvbiB0byByZXNoYXBlIHRoZSBgcmAgZGF0YS4gS2VlcCB0aGUgYmFzZWxpbmUgZGF0YSBpbiBtaW5kIGFzIHdlJ2xsIHdhbnQgdG8gZG8gdGhlIHNhbWUgdGhpbmcgd2l0aCB0aGUgYmFzZWxpbmUgZGF0YSB0byBtYWtlIHRoZW0gbWF0Y2guCgpgYGB7cn0KciA8LSByICU+JSByZW5hbWUoCiAgd2hpY2hfY3JvcF8xXzE2YSA9IHdoaWNoX2Nyb3BfMTZhXzEsCiAgd2hpY2hfbWFpemVfc2VlZF8xXzE2YSA9IHdoaWNoX21haXplX3NlZWRfMTZhXzEsCiAgd2hpY2hfY3JvcF8yXzE2YSA9IHdoaWNoX2Nyb3BfMTZhXzIsCiAgd2hpY2hfbWFpemVfc2VlZF8yXzE2YSA9IHdoaWNoX21haXplX3NlZWRfMTZhXzIsCiAga2dfc2VlZF92ZWdfMV8xNmEgPSBrZ19zZWVkX3ZlZ18xNmFfMSwKICBrZ19zZWVkXzFfMTZhID0ga2dfc2VlZF8xNmFfMSwKICBrZ19zZWVkXzJfMTZhID0ga2dfc2VlZF8xNmFfMiwKICBrZ195aWVsZF8xXzE2YSA9IGtnX3lpZWxkXzE2YV8xLAogIGtnX3lpZWxkXzJfMTZhID0ga2dfeWllbGRfMTZhXzIsCiAgeWllbGRfY29tcGFyZV8xXzE2YSA9IHlpZWxkX2NvbXBhcmVfMTZhXzEsCiAgeWllbGRfY29tcGFyZV8yXzE2YSA9IHlpZWxkX2NvbXBhcmVfMTZhXzIsCiAgCiAgd2hpY2hfY3JvcF8xXzE2YiA9IHdoaWNoX2Nyb3BfMTZiXzEsCiAgd2hpY2hfbWFpemVfc2VlZF8xXzE2YiA9IHdoaWNoX21haXplX3NlZWRfMTZiXzEsCiAgd2hpY2hfY3JvcF8yXzE2YiA9IHdoaWNoX2Nyb3BfMTZiXzIsCiAgd2hpY2hfbWFpemVfc2VlZF8yXzE2YiA9IHdoaWNoX21haXplX3NlZWRfMTZiXzIsCiAgI2tnX3NlZWRfdmVnXzFfMTZhID0ga2dfc2VlZF92ZWdfMTZhXzEsCiAga2dfc2VlZF8xXzE2YiA9IGtnX3NlZWRfMTZiXzEsCiAga2dfc2VlZF8yXzE2YiA9IGtnX3NlZWRfMTZiXzIsCiAga2dfeWllbGRfMV8xNmIgPSBrZ195aWVsZF8xNmJfMSwKICBrZ195aWVsZF8yXzE2YiA9IGtnX3lpZWxkXzE2Yl8yLAogIHlpZWxkX2NvbXBhcmVfMV8xNmIgPSB5aWVsZF9jb21wYXJlXzE2Yl8xLAogIHlpZWxkX2NvbXBhcmVfMl8xNmIgPSB5aWVsZF9jb21wYXJlXzE2Yl8yCikKCgoKYVNlYXNvbiA8LSBuYW1lcyhyKVtncmVwKCIoMS5hKSIsIG5hbWVzKHIpKV0KYlNlYXNvbiA8LSBuYW1lcyhyKVtncmVwKCIoMS5iKSIsIG5hbWVzKHIpKV0Kc2Vhc29uYWxWYXJzIDwtIGMoYVNlYXNvbiwgYlNlYXNvbiwgInNhbXBsZV9pZCIpCmZhcm1lclZhcnMgPC0gYyhuYW1lcyhyKVshbmFtZXMocikgJWluJSBzZWFzb25hbFZhcnNdLCAic2FtcGxlX2lkIikKYGBgCgoKYGBge3J9CiMgZXhhbXBsZSBkYXRhCiMgZGYgPC0gZGF0YS5mcmFtZSgKIyAgIGlkID0gMToxMCwKIyAgIHRpbWUgPSBhcy5EYXRlKCcyMDA5LTAxLTAxJykgKyAwOjksCiMgICBRMy4yLjEuID0gcm5vcm0oMTAsIDAsIDEpLAojICAgUTMuMi4yLiA9IHJub3JtKDEwLCAwLCAxKSwKIyAgIFEzLjIuMy4gPSBybm9ybSgxMCwgMCwgMSksCiMgICBRMy4zLjEuID0gcm5vcm0oMTAsIDAsIDEpLAojICAgUTMuMy4yLiA9IHJub3JtKDEwLCAwLCAxKSwKIyAgIFEzLjMuMy4gPSBybm9ybSgxMCwgMCwgMSkKIyApCiMgCiMgZGYgJT4lCiMgICBnYXRoZXIoa2V5LCB2YWx1ZSwgLWlkLCAtdGltZSkgJT4lCiMgICBleHRyYWN0KGtleSwgYygicXVlc3Rpb24iLCAibG9vcF9udW1iZXIiKSwgIihRLlxcLi4pXFwuKC4pIikgJT4lCiMgICBzcHJlYWQocXVlc3Rpb24sIHZhbHVlKQpgYGAKCmBgYHtyfQpzb3VyY2UoIi4uL29hZmxpYi9taXNjLlIiKQojIGFEYXQgPC0gclssbmFtZXMocikgJWluJSBhU2Vhc29uXSAjIHdvcmtzIGZvciB0aGlzIHRvbyEKIyBhRGF0IDwtIGFEYXRbLGdyZXAoIjE2YV8xIiwgbmFtZXMoYURhdCkpXSAjIHdvcmtzIGZvciB0aGlzCmFEYXQgPC0gclssbmFtZXMocikgJWluJSBzZWFzb25hbFZhcnNdICMgd29ya3MgZm9yIHRoaXMhCgojaHR0cDovL3N0YWNrb3ZlcmZsb3cuY29tL3F1ZXN0aW9ucy8yNTkyNTU1Ni9nYXRoZXItbXVsdGlwbGUtc2V0cy1vZi1jb2x1bW5zCnNlYXNvbmFsRGF0IDwtIGFEYXQgJT4lCiAgZ2F0aGVyKGtleSwgdmFsdWUsIC1zYW1wbGVfaWQpICU+JQogIHRpZHlyOjpleHRyYWN0KGtleSwgYygidmFyaWFibGUiLCAic2Vhc29uIiksICIoXi4qXFxfMS4pKC4pIikgJT4lCiAgbXV0YXRlKHNlYXNvbiA9IHBhc3RlMCgiMTYiLCBzZWFzb24pKSAlPiUgCiAgc3ByZWFkKHZhcmlhYmxlLCB2YWx1ZSkKCm5hbWVzKHNlYXNvbmFsRGF0KSA8LSBnc3ViKCJfMTYiLCAiIiwgbmFtZXMoc2Vhc29uYWxEYXQpKQoKYGBgCgpUT0RPKG1hdHRsb3dlcykgLSBjb25maXJtIHRoYXQgdGhlIHRpZHlyIHByb2Nlc3Mgd29ya2VkIGFzIEkgZXhwZWN0ZWQgYXMgdGhlcmUgYXJlIG51bWVyb3VzIG1pc3NpbmcgdmFsdWVzLiBUaGVzZSBzZWVtIHRvIGFwcGVhciB3aGVyZSB0aGUgdmFyaWFibGUgb25seSBoYWQgb25lIHZlcnNpb24gb2YgdGhlIHZhcmlhYmxlLCBfMTYsIHJhdGhlciB0aGFuIGEgXzE2YSBhbmQgYSBfMTZiLiBDaGVjayBvdXQgaG93IHRoaXMgaXMgaGFuZGxpbmcgdmFyaWFibGVzIHdpdGggXzE3IGluc3RlYWQgb2YgXzE2LgoKIyMgTWVyZ2Ugc2Vhc29uYWwgYW5kIGRlbW9ncmFwaGljIGRhdGEKCmBgYHtyfQpycyA8LSBsZWZ0X2pvaW4oc2Vhc29uYWxEYXQsIHJbLGMobmFtZXMocilbIW5hbWVzKHIpICVpbiUgc2Vhc29uYWxWYXJzXSwic2FtcGxlX2lkIildLCBieT0ic2FtcGxlX2lkIikKYGBgCgojIyBDcmVhdGUgbmV3IHZhcmlhYmxlcwoKIyMjIEZpZWxkIHZhcmlhYmxlcwoKYGBge3J9CnJzJGRpbSA8LSBycyRmaWVsZF9sZW5ndGggKiBycyRmaWVsZF93aWR0aAppbnB1dFZhcnMgPC0gbmFtZXMocnMpW2dyZXAoImZlcnRffHF1YWxpdHlfY29tcG9zdHx0eXBlX2NvbXBvc3R8d2hpY2hfY3JvcHx3aGljaF9tYWl6ZSIsIG5hbWVzKHJzKSldCgpyc1ssaW5wdXRWYXJzXSA8LSBzYXBwbHkocnNbLCBpbnB1dFZhcnNdLCB0b2xvd2VyKQoKIyBpbnB1dCBxdWFuaXRpdGVzCnJzJGZlcnRfa2dfdXJlYTEgPC0gaWZlbHNlKHJzJGZlcnRfdHlwZTE9PSJ1cmVhIiwgcnMkZmVydF9rZzEsIE5BKQpycyRmZXJ0X2tnX3VyZWEyIDwtIGlmZWxzZShycyRmZXJ0X3R5cGUyPT0idXJlYSIsIHJzJGZlcnRfa2cyLCBOQSkKcnMkZmVydF90b3RhbF91cmVhIDwtIGFwcGx5KHJzWywgZ3JlcCgiKHVyZWEuKSIsIG5hbWVzKHJzKSldLCAxLCBmdW5jdGlvbih4KXsKICBzdW0oYXMubnVtZXJpYyh4KSwgbmEucm09VCl9KQoKCnJzJGZlcnRfa2dfZGFwMSA8LSBpZmVsc2UocnMkZmVydF90eXBlMT09ImRhcCIsIHJzJGZlcnRfa2cxLCBOQSkKcnMkZmVydF9rZ19kYXAyIDwtIGlmZWxzZShycyRmZXJ0X3R5cGUyPT0iZGFwIiwgcnMkZmVydF9rZzIsIE5BKQpycyRmZXJ0X3RvdGFsX2RhcCA8LSBhcHBseShyc1ssIGdyZXAoIihkYXAuKSIsIG5hbWVzKHJzKSldLCAxLCBmdW5jdGlvbih4KXsKICBzdW0oYXMubnVtZXJpYyh4KSwgbmEucm09VCl9KQoKCnJzJGZlcnRfa2dfMTducGsxIDwtIGlmZWxzZShycyRmZXJ0X3R5cGUxPT0ibnBrLTE3IiwgcnMkZmVydF9rZzEsIE5BKQpycyRmZXJ0X2tnXzE3bnBrMiA8LSBpZmVsc2UocnMkZmVydF90eXBlMj09Im5way0xNyIsIHJzJGZlcnRfa2cyLCBOQSkKcnMkZmVydF90b3RhbF9ucGsxNyA8LSBhcHBseShyc1ssIGdyZXAoIigxN25way4pIiwgbmFtZXMocnMpKV0sIDEsIGZ1bmN0aW9uKHgpewogIHN1bShhcy5udW1lcmljKHgpLCBuYS5ybT1UKX0pCgoKcnMkZmVydF9rZ18yMm5wazEgPC0gaWZlbHNlKHJzJGZlcnRfdHlwZTE9PSJucGstMjIiLCBycyRmZXJ0X2tnMSwgTkEpCnJzJGZlcnRfa2dfMjJucGsyIDwtIGlmZWxzZShycyRmZXJ0X3R5cGUyPT0ibnBrLTIyIiwgcnMkZmVydF9rZzIsIE5BKQpycyRmZXJ0X3RvdGFsX25wazIyIDwtIGFwcGx5KHJzWywgZ3JlcCgiKDIybnBrLikiLCBuYW1lcyhycykpXSwgMSwgZnVuY3Rpb24oeCl7CiAgc3VtKGFzLm51bWVyaWMoeCksIG5hLnJtPVQpfSkKCgpycyRmZXJ0X2tnXzI1NTVucGsxIDwtIGlmZWxzZShycyRmZXJ0X3R5cGUxPT0ibnBrMjU1NSIsIHJzJGZlcnRfa2cxLCBOQSkKcnMkZmVydF9rZ18yNTU1bnBrMiA8LSBpZmVsc2UocnMkZmVydF90eXBlMj09Im5wazI1NTUiLCBycyRmZXJ0X2tnMiwgTkEpCnJzJGZlcnRfdG90YWxfbnBrMjU1NSA8LSBhcHBseShyc1ssIGdyZXAoIigyNTU1bnBrLikiLCBuYW1lcyhycykpXSwgMSwgZnVuY3Rpb24oeCl7CiAgc3VtKGFzLm51bWVyaWMoeCksIG5hLnJtPVQpfSkKCiNsaW1lCnJzJGxpbWVfb3V0c2lkZSA8LSBpZmVsc2UocnMkZF9saW1lPT0ibGltZV9vdXRzaWRlIiwgcnMka2dfbGltZSwgTkEpCnJzJGxpbWVfdHVidXJhIDwtIGlmZWxzZShycyRkX2xpbWU9PSJsaW1lX3R1YnVyYSIsIHJzJGtnX2xpbWUsIE5BKQpycyRsaW1lX2JvdGggPC0gaWZlbHNlKHJzJGRfbGltZT09ImJvdGhfdHVidXJhX25vbl90dWJ1cmEiLCBycyRrZ19saW1lLCBOQSkKYGBgCgojIyMgRGVtb2dyYXBoaWMgdmFyaWFibGVzCgpgYGB7cn0KcnMkc2Vhc29uXzE2YSA8LSBpZmVsc2UoZ3JlcGwoIjE2YSIsIHJzJG5fdHVidXJhX3NlYXNvbiksIDEsIDApCnJzJHNlYXNvbl8xNmIgPC0gaWZlbHNlKGdyZXBsKCIxNmIiLCBycyRuX3R1YnVyYV9zZWFzb24pLCAxLCAwKQpycyRzZWFzb25fMTdhIDwtIGlmZWxzZShncmVwbCgiMTdhIiwgcnMkbl90dWJ1cmFfc2Vhc29uKSwgMSwgMCkKcnMkbm90Q2xpZW50M1NlYXNvbnMgPC0gaWZlbHNlKGdyZXBsKCJub3RfYV9jbGllbnQiLCBycyRuX3R1YnVyYV9zZWFzb24pLCAxLCAwKQoKYGBgCgojIyBDb21iaW5lIGxvbmcgd2l0aCBiYXNlbGluZQoKVGhlIGBtYXRjaFJvdW5kc2AgZnVuY3Rpb24gdXBkYXRlcyB2YXJpYWJsZSBuYW1lcyBhY3Jvc3Mgcm91bmRzIGFuZCByZXBvcnRzIHRoZSBpbmRleCBhbmQgbmV3IG5hbWUgb2YgdGhlIHZhcmlhYmxlcy4gSSBjYW4gdGhlbiB0YWtlIHRoZSBmaXJzdCBwYXJ0IG9mIHRoZSBsaXN0IGZvciBgZGF0MWAgYW5kIHRoZSBzZWNvbmQgcGFydCBmb3IgYGRhdDJgLgoKYGBge3IsIGV2YWw9Rn0KbWF0Y2hSb3VuZHMgPC0gZnVuY3Rpb24oZGF0MSwgZGF0MiwgdmFyMSwgdmFyMiwgbmV3PU5VTEwsIGNob2ljZT0iZmlyc3QiKXsKICAKICAKICAKICBpZiAoY2hvaWNlPT0iZmlyc3QiKXsKICAgIHZhcjJuZXcgID0gdmFyMQogICAgI25hbWVzKGRhdDIpW25hbWVzKGRhdDIpPT12YXIyXSA8LSB2YXIybmV3CiAgICByZXR1cm4obGlzdCgKICAgICAgbGlzdCh2YXIxLCBncmVwKHZhcjEsIG5hbWVzKGRhdDEpKSksCiAgICAgIGxpc3QodmFyMm5ldywgZ3JlcCh2YXIyLCBuYW1lcyhkYXQyKSkpCiAgICAgICAgICAgICAgICApKQogICAgCiAgfSBlbHNlIGlmIChjaG9pY2U9PSJzZWNvbmQiKSB7CiAgICB2YXIxbmV3ID0gdmFyMgogICAgI25hbWVzKGRhdDEpW25hbWVzKGRhdDEpPT12YXIxXSA8LSB2YXIxbmV3CiAgICByZXR1cm4obGlzdCgKICAgICAgbGlzdCh2YXIxbmV3LCBncmVwKHZhcjEsIG5hbWVzKGRhdDEpKSksCiAgICAgIGxpc3QodmFyMiwgZ3JlcCh2YXIyLCBuYW1lcyhkYXQyKSkpCiAgICAgICAgICAgICAgICApKQogICAgCiAgfSBlbHNlewogICAgdmFyMW5ldyA9IHZhcjJuZXcgPSBuZXcKICAgICNuYW1lcyhkYXQyKVtuYW1lcyhkYXQyKT09dmFyMl0gPC0gdmFyMm5ldyAKICAgICNuYW1lcyhkYXQxKVtuYW1lcyhkYXQxKT09dmFyMV0gPC0gdmFyMW5ldwogICAgcmV0dXJuKGxpc3QoCiAgICAgIGxpc3QodmFyMW5ldywgZ3JlcCh2YXIxLCBuYW1lcyhkYXQxKSkpLAogICAgICBsaXN0KHZhcjJuZXcsIGdyZXAodmFyMiwgbmFtZXMoZGF0MikpKQogICAgICAgICAgICAgICAgKSkKICB9Cn0gCgoKbmFtZXNUb1VwZGF0ZSA8LSByYmluZCgKIGMoImRlbW9ncmFwaGljZGF0ZSIsICJkYXRlIiwgImZpcnN0IiksCiAgYygic2FtcGxlIiwgImRfc2FtcGxlIiwgInNlY29uZCIpCikKCgojIGV4YW1wbGUKZGF0MT1iCmRhdDI9cgp2YXIxID0gImZpZWxkX2RpbTEiCnZhcjIgPSAiZmllbGRfbGVuZ3RoIgpjaG9pY2U9ImZpcnN0IgoKdGVzdCA8LSBtYXRjaFJvdW5kcyhiLCByLCAiZmllbGRfZGltMSIsICJmaWVsZF9sZW5ndGgiLCBjaG9pY2U9ImZpcnN0IikKdGVzdDIgPC0gbWF0Y2hSb3VuZHMoYiwgciwgImZpZWxkX2RpbTIiLCAiZmllbGRfd2lkdGgiLCBjaG9pY2U9ImZpcnN0IikKCgp0ZXN0IDwtIGxhcHBseShuYW1lc1RvVXBkYXRlLCBmdW5jdGlvbih4LHkseil7CiAgdmFsID0gbWF0Y2hSb3VuZHMoYiwgciwgeCwgeSwgY2hvaWNlPXopCiAgcmV0dXJuKHZhbCkKfSkKCmBgYAoKCiMgQW5hbHlzaXMKCiMjIERlbW9ncmFwaGljIHN1bW1hcnkKCiMjIFNvaWwgc3VtbWFyeQoKIyMgTG9uZ2l0dWRpbmFsIHNvaWwgc3VtbWFyeQoKIyMgTWF0Y2hpbmcKCiMjIFJlZ3Jlc3Npb25zCgojIFN1bW1hcnkKCiMjIENoYW5nZXMgdG8gdGhlIHN1cnZleQoKIyBBcHBlbmRpeAoKCg==